Skip to main content
Solved

Child process is stuck on queue when using FMEServerJobSubmitter

jnknnn
Contributor
Forum|alt.badge.img+5

I have a batch workspace that spawns multiple child workspaces using four separate FMEServerJobsubmitters. I have two engines in FME FLow but the other engine is occupied with other processes so I'm able to use only one engine.

 

As i understood, if i set Wait for jobs to complete to "Yes" and Submit Jobs "In Sequence", FME Flow should spawn a sub-engine that will run the child workflows and once they are ready it will continue the parent process. This is the behaviour that I would need because I only want to continue parent workflow when all the child processes are successful. However, with this setup the first child process enters queue and never starts. It also shows in the parent log file that the submit was successful, but it ends up into queue.

 

I have separate queues for the two engines and both the parent process and the child process are tied to engine2 running in queue2 so it should interfere with the other engine and queue. However, it seems that the parent workspaceis blocking the child workspace from running.

 

I Tried to troubleshoot with these https://community.safe.com/s/article/fme-server-troublshooting-fmeserverjobsubmitter#QUEUED

instructions but didn't find anything i could check anymore. Have I understood

the FMEServerJobSubmitter parameters wrong somehow?

Best answer by jnknnn

I was able to solve this by using default queue in FMEServerJobSubmitter and not assigning engine to the default queue. For some reason using any custom queue assigned for specific engine didn’t work. 

View original
Did this help you find an answer to your question?

5 replies

daveatsafe
Safer
Forum|alt.badge.img+19
  • daveatsafeSafer
  • Safer
  • August 21, 2023

Hi @jnknnn​,

If you have the option Submission Mode - Submit Jobs in the FMEServerJobSubmitter, please try setting it to In Sequence. According to the documentation, this should allow the workspace to proceed even though only one engine is available.

jnknnn
Contributor
Forum|alt.badge.img+5
  • jnknnnContributor
  • Author
  • Contributor
  • August 22, 2023
daveatsafe wrote:

Hi @jnknnn​,

If you have the option Submission Mode - Submit Jobs in the FMEServerJobSubmitter, please try setting it to In Sequence. According to the documentation, this should allow the workspace to proceed even though only one engine is available.

I have set submission mode to In Sequence

image

daveatsafe
Safer
Forum|alt.badge.img+19
  • daveatsafeSafer
  • Safer
  • August 22, 2023
jnknnn wrote:

I have set submission mode to In Sequence

image

I did a quick test on FME Flow 2023 with only one engine activated, and a workspace with four FMEFlowJobSubmitters worked OK, both in parallel and in series.

However, we have had problems in past versions of FME Server if the child workspace is set to use a different Python version than the parent, or if the child workspace crashes during its run.

 

What version of FME Server are you using?

 

Would you mind sharing the workspace with the FMEServerJobSubmitter in it, and the log file from its run?

jnknnn
Contributor
Forum|alt.badge.img+5
  • jnknnnContributor
  • Author
  • Contributor
  • August 28, 2023
jnknnn wrote:

I have set submission mode to In Sequence

image

We are using Fme server  2021.2.4. and workspaces are made in desktop 2021.2.1.0. I checked that all workspaces should be using python 3.8+. Parent workspace is attached, I was not able to access log-file anymore. However, the last line in the log file was "FMEServerJobSubmitter_3 (ServerFactory): <fmeserverhost>- Running workspace '<workspacename>' in repository '<repositoryname>'... and the first child workspace entered queue.

 

I removed sql's, connection parameters and other connection information for ip and other obvious reasons but other than that it should work as intended. In all FMEServerJobSubmitter transformers i had connection parameters embedded.

jnknnn
Contributor
Forum|alt.badge.img+5
  • jnknnnContributor
  • Author
  • Contributor
  • Best Answer
  • March 28, 2024

I was able to solve this by using default queue in FMEServerJobSubmitter and not assigning engine to the default queue. For some reason using any custom queue assigned for specific engine didn’t work. 

Reply


Helpful Members This Week

Community Stats

30,974
Posts
117,398
Replies
38,759
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.