Solved

How to get the FMEServerJobWaiter to wait until the last server job is complete?

  • 6 October 2017
  • 2 replies
  • 14 views
M
Badge +3

I have several hundred jobs that I'm processing on FMECloud. The FMEJobSubmitter is set to 'Wait for job to complete: No" so I can take full advantage of 10 engines and the queue. After the JobSubmitter I have a FeatureHolder, then a Sorter (Descending by _Job_id), then a Sampler (First 1 Features) to get the last job "_job_id". When the last jobs arrive from the queue and run on the 10 engines the dataset for the last job is smaller and therefore finishes before the others, therefore the FMEServerJobWaiter sees the last job is Complete the workbench continues, although all the jobs are not really complete. If one could set the Priority in the FMEServerJobSubmitter via an attribute one could set the priority of the last job to be lower (ie. 101) than the others (ie. 100) forcing it to really be last.

icon

Best answer by stephen.wong 6 October 2017, 04:25

View original

2 replies

S

Hi mikek, in 2017.0 you can take advantage of the option 'Submit Jobs: In Parallel', followed by 'Wait for Job to Complete: Yes'. What this will do is submit your several hundred jobs at once, and wait until all of them are complete, and it doesn't depend on the order in which the jobs finish.

Read more about this option in the documentation:

http://docs.safe.com/fme/2017.1/html/FME_Desktop_Documentation/FME_Transformers/Transformers/fmeserverjobsubmitter.htm

Hope this works!

M
Badge +3

Hi mikek, in 2017.0 you can take advantage of the option 'Submit Jobs: In Parallel', followed by 'Wait for Job to Complete: Yes'. What this will do is submit your several hundred jobs at once, and wait until all of them are complete, and it doesn't depend on the order in which the jobs finish.

Read more about this option in the documentation:

http://docs.safe.com/fme/2017.1/html/FME_Desktop_Documentation/FME_Transformers/Transformers/fmeserverjobsubmitter.htm

Hope this works!

Thanks stephenwong1331, I was on an older release of 2017.1, I updated FME to the most recent release and the 'Submit Jobs: In Parallel' appeared!

 

 

Reply


Community Stats

28,692
Posts
109,433
Replies
36,544
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.