Question

How to use Workspace Runner on FME Server without using 2 Engines

  • 17 January 2017
  • 8 replies
  • 27 views
J
Badge

I have a set of workspaces which I need to run in FME Server, a process which needs to be repeated every few weeks.

I have set up a workspace which contains a set of FMEServerJobSubmitter transformers, with 'Wait for Server Job to Complete' set to Yes. This works fine for running on my Desktop, and uses up 1 FME Server Engine to run through the jobs.

However, I would like to do the whole process in FME Server, not using up an FME Desktop license. When I run the FMEServerJobSubmitter workspace in FME Sever, 2 engines are used up - one for the workspace runner and another for actually running the jobs.

Is there a way around this?

Thanks

8 replies

redgeographics
Rank
Userlevel 5
Badge +25

There is if you have an FME Server with just one engine, but other than that I think this is expected behaviour.

itay
Rank
Badge +16

Yes, you can run the first child workspace based on a schedule (or other trigger) and trigger the following child workspaces via the notification service.

Hope this helps.

david_r
Rank
Userlevel 4

Unlike the others I think there is an error here, it should be possible to chain jobs synchronously on a single engine. From the documentation:

When Wait for Server Job to Complete is set to Yes, and multiple FMEServerJobSubmitter transformers are linked to manage the workflow of multiple jobs on the same FME Server, all jobs run as child processes of the same FME engine that is engaged by FME Server to run the first job. No other FME engine is engaged. As a result, child jobs do not appear in the FME Server job queue or job history, and do not have their own job_id's. Additionally, all job logs are included in the job log of the parent (controller) job.

The problem is that some conditions will (silently) force FME Server into running your child jobs asynchronously, i.e. on a separate engine. One particularly sneaky trap is if you e.g. use "localhost" inside your FMEServerJobSubmitter or a different host name as the one used by your parent workspace.

You should be looking into your fmeserver.log and fmeprocessmonitorengine.log files where you'll usually find a warning about FME Server forcing your child workspace into asynchronous mode. Post whatever you find here and hopefully we'll be able to help you further.

runneals
Rank
Badge +8

The other big issue that I ran into is getting the host name right. In our setup, I figured it had to be 1 of 3 names... the Server name (FMEPROD), the url to access it externally/internally (httpS://fmeurl.com), or the internal url (fmeurl.com). If you don't have this set up correctly, it will take multiple engines to run. It ended up being fmeurl.com, however you can't hit that url directly, so we use the machine name to connect to the server and then we paste in the internal url into the transformer attributes.

steveatsafe
Rank
Badge +11

What version of FME Server are you using @jess11? Older versions of FME Server did experience issues with child processes. Hopefully you are on a more recent version of FME Server and this will be possible for you!

Great feedback from @david_r & @runneals on configurations that could force a new engine (additional fme engine license required) to be used instead of a new child process being invoked (doesn't use an additional fme engine license).

SteveatSafe
J
Badge

Thanks to everyone for their advice, as @david_r and @runneals suggested, the problem was due to the JobSubmitter Parent workspace being a different Host Name from the Child workspaces.

The solution I used was to set the [FME_SERVER_HOST] parameter, under FME Server Parameters, to the internal URL. Then in the Connection Parameters for each JobSubmitter, I used this parameter as the FME Server URL. The workspaces can now run one at a time and using just one engine.

david_r
Rank
Userlevel 4

Thanks to everyone for their advice, as @david_r and @runneals suggested, the problem was due to the JobSubmitter Parent workspace being a different Host Name from the Child workspaces.

The solution I used was to set the [FME_SERVER_HOST] parameter, under FME Server Parameters, to the internal URL. Then in the Connection Parameters for each JobSubmitter, I used this parameter as the FME Server URL. The workspaces can now run one at a time and using just one engine.

That is great to hear, thanks for sharing your solution.
frro
Rank
Badge +6

I just struggled a bit with the issue of running a job using FMEServerJobSubmitter on a single-engine FME Server (2016.2.1). I had a master workspace that would start a sub-workspace and the wait for it to finish, before carrying on with the translation in the master wokspace. My findings were that you need to get the machine name casing correct when you set the URL Parameter in the FMEServerJobSubmitter.

  • If I had the casing wrong the FMEServerJobSubmitter the sub-workspace would get queued in FME Server, waiting for a free engine.
  • If I had the casing correct in the FMEServerJobSubmitter the sub-workspace would run within the master workspace's process.

To find the correct casing you can have a look in any FME Server job log file, on row 5 the Machine Host Name is written. Copy the Machine Host Name into the FMEServerJobSubmitter URL parameter and you should be fine.

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.