Question

Automation Workspace behaving differently to Workbench

kieranmg
Badge +3

Hi All, I have a strange one i think!

 

Its a simple workspace which takes a Spatial SQL table of property boundaries and reprojects it before writing to a mapinfo tab file. Running the workspace on workbench (same server) gives the correct results, running it on fme server produces a tab file which is from what i can see 99% correct, but muddled up in places.

 

  • Confirmed Server workspace and Workbench are identical
  • Deleted server workspace and published from workbench again - same results
  • Server job log doesn't give any indications to any issues - features written are same
  • Restarting server does nothing (Express installation)
  • workbench & server are same version 2022

 

Heres the weird bit!, if i inspect the tab file after running the workspace on fme server it seems to have swapped the attributes of about 13 spatial features around and muddled them about. Screenshots might give a better idea.

Expected resultsOrder changed when inspecting tabAttributes have also shifted into another columnAny suggestions are appreciated!

2 replies

S

Whenever data shifts columns, I normally suspect unescaped characters in the data that are misintepreted as special characters/control characters, such as column dividers, new lines/line feed, etc.

 

What happens if you make a Tester or TestFilter, and you test every single attribute/column and its value against a regular expression that blacklist everything except the characters you expect in your data? For instance the expression below should work because it will whitelist only the following characters (that I see in your data in the pictures you provided):

  • letters a-z and A-Z
  • numbers 0-9
  • comma
  •  punctuation
  • hyphen
[^a-zA-Z0-9,.-]

Any features that fullfills the test (is not unfiltered) could possibly contain special control characters and possibly non-renderable characters that could be the reason the writer to mangles the data.

 

If that happens to be the case, you can try to sanitize the affected data by putting the same expression into a StringSearcher, and replace whatever the regular expression matches with an empty string ("").

 

Edit: Another way to check if it is a certain feature causing all the issue, is to sort your features by UniqueID prior to writing. When you inspect the file and find the shift in columns, copy/paste the UniqueID for the last feature that was OK, and the first feature that is not OK. Go back to your workspace and filter out these two features specifically and see if columns still shift. If the shifting disappears, one of the two filtered features will be the culprit.

kieranmg
Badge +3

Thanks @skjalg​  for your assistance.. I tried both suggestions but the results were inconclusive. There is obviously some way fme server is interpreting the data differently from workbench.

 

I tried removing basically all the attributes i could with a attributekeeper before writing, whilst still keeping the bare minimum to identify each feature. The idea was to test each attribute to narrow it down to a single column. Having done that i noticed that values of attributes i had removed were still showing up in the kept columns in the tab file. So the issue had to be earlier in the workspace before the keeper.

 

Long story short, i had a nullattributemapper setting null values to Empty Strings from an issue i had in the past. Removing this resolved the issue, I wanted to blame the tab file format but i'm not so sure it was the problem. My guess is fme server has shifted attributes values after the NullAttributeMapper for whatever reason.

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.