Skip to main content
  • EN

Hello,

I'm not able to figure out how to run child jobs using FMEServerJobSubmitter based on a list.

What I have done is a "master" fme workspace which spawn "jobs" passing parameters to instances of another workspace. It's done based on list so there is a list - and if there are 10 elements in the list 10 jobs will be submitted to the server. It means 2 available engines will run the jobs and the rest will be queued. It's undesired situation, since this will run on a 6 engine FME server and the current solution will use all of the available engines. It's undesired because estimation of this job running on 1 engine is 10 days (it's large raster). So it would block all other jobs for quite amount of time - on 6 engines obviously it would take less time like 1-2 days, however it's still not acceptable.

 

So the question - is it possible to limit the FMEServerJobSubmitter to be running only in 2 instances at once, so looping the list - taking 2 (or N) elements, running job submitters for these elements waiting until the jobs finish and then taking another element from list.

 

Thanks for help

Tomasz

Hi @tomasz,

You could set up your FME Server using Job Queues to ensure that only 2 engines are used for actually running the jobs submitted by the FMEServerJobSubmitter in your workflow. That would involve created a new Job Queue from the Engines & Licensing page of FME Server, then assigning 2 of the engines to that queue, then finally adjusting the FMEServerJobSubmitter to submit the jobs it runs to that new queue (that option can be found under the Advanced parameters in the transformer).

Here's a link to a blog post that walks through how Queues work and how they can be useful: https://blog.safe.com/2018/04/queues-and-data-processing-evangelist173/

Hope that helps!

Reply


Community Stats

30,159
Posts
114,480
Replies
37,806
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.