I tried longPoll parameter TRUE and received a 200 for the submitted jobs. Although the response body has no status information.

And the Jobs submitting workspace ended before the submitted jobs. So not really a progress. 

Firstly, I suspect/hope Safe will create a replacement transformer for this tool. So hopefully you wont have to create it yourself.

Secondly, why are you not setting the wait until complete to Yes if you’re also using a Job Waiter anyway?

With respect to your question - you’re on the right track with the REST call.

Because version 3 or f the rest api is being deprecated, however, you should instead be using version 4 of the REST api. You should find the API doc at:
 

HOST/fmeapiv4/docs/index.html

The call you want to use is:
 

GET /fmeapiv4/jobs/{jobid}

In this case the {jobid} should be returned from the FMEServerJobSubmitter. 

You will then need to parse the _reponse_body attribute, you can use a JSONFlattener. The “status” is what you’re looking for here. 

Now comes the tricky part….You need to create a looping transformer to repeat the call until you get a final status.
 

That’s exactly what I try to avoid. FME is still cumbersome if it comes to looping. Http calls with looping would rather end up in a PythonCaller than using the FME transformers.

So I hope (as you do) safe wont leave me/us alone with the JobWaiter deprecation and will provide a replacement for it instead.

thx for feedback

Klaus

I am also very curious to the answer from Safe on this issue, I do hope that Safe will provide a workaround or another solution and wont let us ‘hanging’….also when is the API V4 going to be released? Will Safe let us know in advance? or wil we notice it once our processing is broken?

Hi @itay and @kalbert,

We currently have no plans to replace the JobWaiter.

Have you explored using the Split/Merge block in Automations to accomplish this? Is there any specific functionality that you feel is missing for your use case?

Also, regarding V3, we are still actively working on V4, so there's no announcement yet. For V3 deprecation, I recommend keeping an eye on the FME Deprecations page for any updates: FME Deprecations Page.

Looking forward to hearing more!

Best regards,
Sienna

Hi,

No I have not was hoping that the transformer or a workaround will be made available, can you explain how that can be accomplished with the items you mention.

Hi @itay, 

Thank you for the feedback. I’ve passed it onto our product managers. It’s good to know when deprecated functionality will be missed by users. 

From the product management perspective, the transformer had low usage and we believe Automations offered a better and more efficient workaround than the transformer. However, please let us know if Automations don’t meet your needs. The product development team was curious to know more about your use case.

The workaround in automation might look something like this. This workflow is initiated by a manual trigger, however it could be any other trigger like a schedule or a directory watch. 

From there, 5 workspaces will be initiated to run in parallel. If all the workspace are successful a message will be sent to the checkmark port of the split merge block. If a single action fails, a message will be sent to the “X” port. The best thing about this method is it should be more efficient and won’t require a poll on the Flow instance which caused users issues in the past. 

I’m happy to chat in depth more about your use case and the transition to automations. Just send me a direct message and we can set up a time to discuss. 

Thank you,

Sienna
 

Hi,

I have sent you a direct message to continue this discussion.

Hi @siennaatsafe 

Automation might be a good solution if you start a new ETL project from scratch. For customers running larger ETL programs orchestrated by let’s name it master or controller workspaces doing complex work it’s simply too expensive migrating everything to Automation. 

However we are going to replace the JobWaiter with a custom looped HttpCaller. Reason is that we’ve recognized some unexpected latency of the JobWaiter. We see jobs finished but up to 1h latency before the JobWaiter reacts. Replaced with a looped HttpCaller it’s just a second. (FME Flow 2024.1.)

kind regards

Klaus