I've designed a work flow that makes use of a control bench that subsequently uses the FMEServerJobSubmitter within it multiple times. I've set the wait for completion parameter properly everywhere, and fully qualified server URLs are being used so that jobs run synchronously. When I schedule the one control bench three times (each scheduled job processes a different data set) 00:02, 00:03, 00:04 respectively, I would expect each engine to host an iteration of the control bench. However, at any given time, only one to two engines are being utilized while the third sits dormant. Any ideas or suggestions?
Question
How to fully utilize multiple engines on FME server?
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
Hi
When using the FMEServerJobSubmitter with "Wait for completion" set to YES, all child jobs will run on the same engine, regardless of how many engines are available. To utilize multiple engines you will have to set it to NO (asynchronous) and use the FMEServerJobWaiter with the returned job ID.
From the FMEServerJobSubmitter documentation:
Note: When Wait for Server Job to Complete is set to Yes, and multiple FMEServerJobSubmitter transformers are linked to manage the workflow of multiple jobs on the same FME Server, all jobs run as child processes of the same FME engine that is engaged by FME Server to run the first job. No other FME engine is engaged. As a result, child jobs do not appear in the FME Server job queue or job history, and do not have their own job_id's. Additionally, all job logs are included in the job log of the parent (controller) job. For more information, see "FME Server Workflow Management," at http://knowledge.safe.com/articles/How_To/FME-Server-Workflow-Management.
David
Hi
When using the FMEServerJobSubmitter with "Wait for completion" set to YES, all child jobs will run on the same engine, regardless of how many engines are available. To utilize multiple engines you will have to set it to NO (asynchronous) and use the FMEServerJobWaiter with the returned job ID.
From the FMEServerJobSubmitter documentation:
Note: When Wait for Server Job to Complete is set to Yes, and multiple FMEServerJobSubmitter transformers are linked to manage the workflow of multiple jobs on the same FME Server, all jobs run as child processes of the same FME engine that is engaged by FME Server to run the first job. No other FME engine is engaged. As a result, child jobs do not appear in the FME Server job queue or job history, and do not have their own job_id's. Additionally, all job logs are included in the job log of the parent (controller) job. For more information, see "FME Server Workflow Management," at http://knowledge.safe.com/articles/How_To/FME-Server-Workflow-Management.
David
My issue is not with the FMEServerJobSubmitter transformer. I'm scheduling my control bench from the FME server scheduler to run 3 times (ideally simultaneously) with each bench updating a different dataset. Why doesn't the third scheduling of the control bench use the third engine?
My issue is not with the FMEServerJobSubmitter transformer. I'm scheduling my control bench from the FME server scheduler to run 3 times (ideally simultaneously) with each bench updating a different dataset. Why doesn't the third scheduling of the control bench use the third engine?
So if the scheduler starts 3 different instances of your workspace, does the third instance go into the job queue while the third engine sits idle?
So if the scheduler starts 3 different instances of your workspace, does the third instance go into the job queue while the third engine sits idle?
That is correct.
That is correct.
Check that you haven't (accidentally) activated job routing. Also check that you have 3 licensed engines (Manage / Administration / Engine management) running.
If that doesn't work, you'll have to fine comb the server logs to see what is going on.
+59- Safer
That is correct.
As David R says, I see no reason why a job would be queued with an engine idle. So check what he suggests and if the issue is not obvious please do contact the Safe support team.
+59- Safer
Anyone else have any idea what is preventing the third engine from activating? Thanks
Did you check the number of licensed engines and that the third engine wasn't processing anything else at the same time? Also, what version of FME are you using? The only other similar case I could find was this (https://knowledge.safe.com/articles/312/licensing-error-when-using-fmeserverjobsubmitter.html) - it doesn't sound to me like it's the same thing, but I thought I'd mention it. If none of that helps, I really think you should contact the support team (safe.com/support) for help
+59- Safer
Anyone else have any idea what is preventing the third engine from activating? Thanks
I do wonder if it's related to a prior case you had (C110555) so I've asked the same support person (Steve) to look at this question here.
+13- Safer
I do wonder if it's related to a prior case you had (C110555) so I've asked the same support person (Steve) to look at this question here.
@davidmelancon This is a strange one David. We'll have to connect and see what happening. I've sent you a message. We can followup back here once we learn the resolution.