Have you verified in the logs that both the master and the child workspace actually runs on the same engine?
A common issue is if the FMEServerJobSubmitter references a different host name than used by the master workspace, FME Server will spin up the job on a separate engine, even if both host names are physically referencing the same server.
Typical example:
Master workspace runs on my.fmeserver.local, but the FMEServerJobSubmitter connects to localhost. The child workspace will then be forced to run on a separate engine.
I would also recommend that you consider upgrading FME Server, there has been a lot of improvements since 2016 in this regard.
Have you verified in the logs that both the master and the child workspace actually runs on the same engine?
A common issue is if the FMEServerJobSubmitter references a different host name than used by the master workspace, FME Server will spin up the job on a separate engine, even if both host names are physically referencing the same server.
Typical example:
Master workspace runs on my.fmeserver.local, but the FMEServerJobSubmitter connects to localhost. The child workspace will then be forced to run on a separate engine.
I would also recommend that you consider upgrading FME Server, there has been a lot of improvements since 2016 in this regard.
Hello,
thank you, Yes I verified it and for all jobs it is the same engine.
Do you know what is the limit of concurent fme processes and the same server? Is it per engine or per server?
Hello,
thank you, Yes I verified it and for all jobs it is the same engine.
Do you know what is the limit of concurent fme processes and the same server? Is it per engine or per server?
One engine = one job. If you let a child workspace run on the same workspace as the master workspace, the master workspace will be paused ("put aside") as long as the child workspace runs. The master workspace will continue once the child workspace has termina
One engine = one job. If you let a child workspace run on the same workspace as the master workspace, the master workspace will be paused ("put aside") as long as the child workspace runs. The master workspace will continue once the child workspace has termina
Ok, but then I get
FME license system failure: The maximum number of concurrent FME instances specified in the license has been reached (-508)
So I guess that even if I have ServerJobSubmitter is set to wait for jobs to complete
new FME.Exe processes so questions is how many FME.exe processes you can have on one FME engine?
Ok, but then I get
FME license system failure: The maximum number of concurrent FME instances specified in the license has been reached (-508)
So I guess that even if I have ServerJobSubmitter is set to wait for jobs to complete
new FME.Exe processes so questions is how many FME.exe processes you can have on one FME engine?
There should (normally) be no instances of fme.exe running on FME Server, only fmeengine.exe, which, confusingly, presents itself as "FME EXE" in the Task Manager.
I suppose you do not have any WorkspaceRunners?
You may also want to contact Safe support for this.
There should (normally) be no instances of fme.exe running on FME Server, only fmeengine.exe, which, confusingly, presents itself as "FME EXE" in the Task Manager.
I suppose you do not have any WorkspaceRunners?
You may also want to contact Safe support for this.
Thanks for hints,
I only use ServerJobSubmitters. And in my case it looks like this:
I sent message to safe support. We will see :)
Hi @witos,
There can be a maximum of 7 child processes kicked off by a job ( for a total of 8 processes). However, in FME 2016 it didn't always do the right thing - have you checked out this article - https://knowledge.safe.com/articles/55369/fme-server-troublshooting-fmeserverjobsubmitter.html which highlights the differences in 2016 and 2017+.
Cheers,
Cat
Hi @witos,
Thank you for contacting Safe Software Support. Now that we have understood the issue I wanted to also update this post on the Forum so that the answer is available publicly for anyone else that may run into this issue.
The reason you are running into this error is because the FME licensing agreement only allows a maximum of eight fme.exe processes to run in parallel.
In FME Server 2016 there are three ways I know of that could cause you to hit this error:
- Running on a single engine your parent workspace contains 9 layers of workspaces with fmeserverjobsubmitters in each. Note only one fme.exe process is launched per workspace no matter how many JobSubmitters it contains.
- Across multiple engines you have workspaces that launch multiple layers of child jobs which combined add up to more than 8.
- If you cancelled a job containing a JobSubmitter the fme.exe process failed to shut down so is taking up part of the total count even though it isn't being used - this was a known issue in 2016.
This is because in 2016 the behaviour mirrored that of FME Desktop and so was monitoring these fme.exe processes on a per machine basis. If you have multiple engines the workaround would be to deploy these on different machines.
In FME Server 2017.1 and newer this has been rectified and is now on a per engine basis. This can be observed in Windows Task Manager as it now launches additional fmeengine.exe processes instead of fme.exe processes to avoid these being counted like for desktop engines.
There were a number of problems with FMEServerJobSubmitter in 2016 which Cat highlighted. However I have also updated the Maximum Concurrent Processes article that covers this error to include the fme.exe per machine limitation that is on 2017 and older.