Question

WorkspaceRunner bonking on Rejected Feature despite settings

  • 4 January 2018
  • 5 replies
  • 10 views
B
Badge

Situation: I have a workspace with a JSONFragmenter transformer that occasionally receives malformed JSON (unescaped special characters). I want it to skip over them so I set the workspace parameter for "Rejected Feature Handling" to "Continue Translation" and all is good.

But when I reference the above workspace using a WorkspaceRunner (Wait for Job to Complete is set to Yes) on both desktop or server, it bonks the parent workspace even though I have the same setting for the Rejected Feature Handling in the parent as well.

So am I missing a setting somewhere?

The band-aid fix is to add a Logger to the Rejected port. But I got a lot of JSONFragmenters and I'm hoping for a more elegant solution.

-- Brian

5 replies

J
Badge +2

Hi @brian_timoney

 

Are you controlling rejected ports in the navigator or in the workbench settings? I'd make sure their set and saved in the navigator.

 

 

There is also a JSONValidator transformer, I don't know whether that'd help you here
B
Badge

in the Navigator I have set Workspace Parameters --> Translation --> Rejected Feature Handling: Continue Translation for both the parent and child workspaces.

I'm aware of the JSONValidator, but this is a case where I'm not concerned about the occasional error enough to catch it and deal with it robustly, but rather ignore the hiccup and keep the workspace running.

fmelizard
Rank
Userlevel 4
Badge +13

To be clear, in both the parent and the child workspace, the Continue Translation option is set in the navigator.

And the child will run fine in Workbench.

But when the parent calls the child using WorkspaceRunner, the WorkspaceRunner errors and causes the parent to shut down?

If that is the case, could you post the relevant chunk of the parent's logfile here? I'm curious what it is saying.

I've mocked up a simple Parent and Child in FME 2017.1 and they seem fine:

parentwithrunner.fmw

childwithrejection.fmw

Now, I did recall something being up with JSONFragmenters in particular and... https://knowledge.safe.com/questions/54446/issue-with-the-jsonfragmenter-in-fme-server-20171.html seems to confirm that. Will ask to team to get back to you with an update as to what the situation was with that particular transformer.

Short answer -- I think you managed to find the one transformer with problems handling rejection in FME 2017... (It might be helpful to know what build you're using as well then...)

deanatsafe
Rank
Badge +6

Thanks for bringing this up. There appears to be a problem with JSONFragmenter in both FME 2017 and FME 2018. I've reproduced your problem here and have filed a bug report. I've also created a support case and will follow up with you there. This only seems to come up when the workspace is run 'headless' - that is with WorkspaceRunner or on FME Server. My guess is that the rejected port caching for inspection is acting like a logger when run interactively on workbench. This does seem to be related to JSON transformers as a similar configuration that Dale posted below using Bufferer rejected features did not fail, whereas JSONExtractor does fail.

 

In the meantime for a workaround, I understand that you would rather not add Loggers to all your JSONFragmenters. One possible way around this is to use a JSONValidator to validate your json document before doing any of your parsing. You could then eliminate any features with invalid json before they get to the JSONFragmenters. Note that a similar problem was fixed recently in FME 2018 for JSONValidator. So you would need to add a Logger to the JSONValidator rejected port unless you are running FME 2018.

 

 

Thanks again for reporting this. We will definitely prioritize testing and fixing this for the other JSON transformers.
B
Badge

Thanks for bringing this up. There appears to be a problem with JSONFragmenter in both FME 2017 and FME 2018. I've reproduced your problem here and have filed a bug report. I've also created a support case and will follow up with you there. This only seems to come up when the workspace is run 'headless' - that is with WorkspaceRunner or on FME Server. My guess is that the rejected port caching for inspection is acting like a logger when run interactively on workbench. This does seem to be related to JSON transformers as a similar configuration that Dale posted below using Bufferer rejected features did not fail, whereas JSONExtractor does fail.

 

In the meantime for a workaround, I understand that you would rather not add Loggers to all your JSONFragmenters. One possible way around this is to use a JSONValidator to validate your json document before doing any of your parsing. You could then eliminate any features with invalid json before they get to the JSONFragmenters. Note that a similar problem was fixed recently in FME 2018 for JSONValidator. So you would need to add a Logger to the JSONValidator rejected port unless you are running FME 2018.

 

 

Thanks again for reporting this. We will definitely prioritize testing and fixing this for the other JSON transformers.
Thanks for your prompt attention to the issue. -- BT

 

 

Reply


Community Stats

28,666
Posts
109,340
Replies
36,535
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.