FME Server submits jobs with delay

Can you share a screenshot of the configuration of the first FMEServerJobSubmitter? I'm assuming that they are configured similarily.

Hi David, please, see below. e attached.

Have you looked in the FME Server logs to see exactly what is happening with the jobs and the engines? In particular, check logs / engine / current / fmeprocessmonitorengine.log

I have checked the logs and this is what I found.

The job 312 finished at 22:25:45, and the 313 started at 22:26:29, which is 44 seconds after the 312 has finished.

Now, let's look at the log:

So, 312 job was submitted at 10:25:44, at 10:25:45 it was finished ant than there was a problem delivering results and some other stuff before job 313 was submitted. Not sure where the problem is.

I have checked the Windows/System32/drivers/etc/host on the server where web application is installed and it is exactly as explained here:

https://knowledge.safe.com/articles/333/request-times-out-network-error-between-tomcat-and.html

I have also noticed that if I go to Licensing & Engines -> Engine page, after a few seconds I get this error.

I did what was explained if you hit the link, but it did not solve the problem. I have a feeling that this may be connected to the issue I am having.

I agree, if there are e.g. internal timeouts it could explain some delays. I think a closer look at the server system logs and network configuration is in order.

I know this is old... but I wanted to share some information that is potentially related to this.

I have recently seen the reported behaviour when a SQL Server AlwaysOn was where the FME Server System Database was hosted, but the multisubnetfailover=true flag was not added to the FME Server Database Connection in the config files of the FME Server. So what was happening was the two cores/engines were interacting with different databases (behind the curtains of AlwaysOn) and this was presenting problems to how the jobs were processed, slowing them down considerably (delaying being pulled from the queue it seems).

The customer, in my case, added the flag and jobs were no longer delayed.

Please review the documentation for Changing the Database Provider.

There is a possibility of other clustered Databases where this could also present a similar behaviour but I have little evidence of this yet and one possible suspect. Please do ensure the DB_JDBC_URL in the FME Server Config file(s) are properly set for the clustered database you are using.

If you have any questions don't hesitate to post a new question or reach out to Safe Software Support.

Another thing that could be happening is if the Core can’t see the Distributed Engine (port 7500) this REST API error will get returned when visiting the Engine Management page.  The core attempts to interrogate the engine manager on the distributed engine host to get the engine counts.  This is important to resolve by ensuring the port is open on the distributed engine host.  A good tool to use in this case is Powershell and test-netconnection command as this will ensure connectivity is possible.  

Open Powershell, and run this from the FME Flow Core system to the distributed engine host

i.e. test-netconnection -computername GISSERVER01DEV -port 7500

If you are experiencing such issues, one way, there’s a good chance the distributed engine host is having some connectivity issues, the other way as well… and this could lead to the FMEFlowJobSubmitter misbehaving.  This transformer will establish connectivity from one Job (parent) to another (child), waiting for results.  The issue in this post seems to suggest port issues, and is likely on the ephemeral ports.

Please review the FME Flow Ports documentation to ensure all necessary ports are open.  

It is important to know that all nodes have proper connectivity via ports that FME Flow will use.  There are always ephemeral ports in play for FME Flow environment, a regular part of network connectivity between systems, and if you have restrictions in your environment (not so common), please ensure to tweak the FME Flow configuration and update the port pool range.