Skip to main content
Question

FME Server submits jobs with delay


Forum|alt.badge.img

Hi everyone,

 

I have a parent workspace that looks like a series of FMEJobSubmitter transformers:

Each FMEServerJobSubmitter set up at below:

Submit Jobs: In Sequence

Wait for the Jobs to Complete: Yes

 

When I call this parent workspace from the Web Interface, something weird is happening. First job gets submitted straight away and it becomes Completed after a second. Then the second one will only appear in Running jobs after about 45 seconds. It is not in the Queued and one of the engines is not doing anything.

The same case is happening for the rest of the workspaces - they will only appear in Running after about 45 seconds once the previous one gets Completed.

It is worth mentioning that we installed our FME server as distributed components (3-tier).

If it is installed as Express, it is working fine.

 

Any help, please?

PS We are using the latest build (20596)

8 replies

david_r
Celebrity
  • July 17, 2020

Can you share a screenshot of the configuration of the first FMEServerJobSubmitter? I'm assuming that they are configured similarily.


Forum|alt.badge.img
  • Author
  • July 17, 2020
david_r wrote:

Can you share a screenshot of the configuration of the first FMEServerJobSubmitter? I'm assuming that they are configured similarily.

Hi David, please, see below. e attached.


david_r
Celebrity
  • July 17, 2020

Have you looked in the FME Server logs to see exactly what is happening with the jobs and the engines? In particular, check logs / engine / current / fmeprocessmonitorengine.log


Forum|alt.badge.img
  • Author
  • July 17, 2020
david_r wrote:

Have you looked in the FME Server logs to see exactly what is happening with the jobs and the engines? In particular, check logs / engine / current / fmeprocessmonitorengine.log

I have checked the logs and this is what I found.

The job 312 finished at 22:25:45, and the 313 started at 22:26:29, which is 44 seconds after the 312 has finished.

Now, let's look at the log:

So, 312 job was submitted at 10:25:44, at 10:25:45 it was finished ant than there was a problem delivering results and some other stuff before job 313 was submitted. Not sure where the problem is.

I have checked the Windows/System32/drivers/etc/host on the server where web application is installed and it is exactly as explained here:

https://knowledge.safe.com/articles/333/request-times-out-network-error-between-tomcat-and.html

 


Forum|alt.badge.img
  • Author
  • July 17, 2020

I have also noticed that if I go to Licensing & Engines -> Engine page, after a few seconds I get this error.

I did what was explained if you hit the link, but it did not solve the problem. I have a feeling that this may be connected to the issue I am having.


david_r
Celebrity
  • July 17, 2020
involver wrote:

I have also noticed that if I go to Licensing & Engines -> Engine page, after a few seconds I get this error.

I did what was explained if you hit the link, but it did not solve the problem. I have a feeling that this may be connected to the issue I am having.

I agree, if there are e.g. internal timeouts it could explain some delays. I think a closer look at the server system logs and network configuration is in order.


steveatsafe
Safer
Forum|alt.badge.img+12
  • Safer
  • December 1, 2021

I know this is old... but I wanted to share some information that is potentially related to this.

 

I have recently seen the reported behaviour when a SQL Server AlwaysOn was where the FME Server System Database was hosted, but the multisubnetfailover=true flag was not added to the FME Server Database Connection in the config files of the FME Server. So what was happening was the two cores/engines were interacting with different databases (behind the curtains of AlwaysOn) and this was presenting problems to how the jobs were processed, slowing them down considerably (delaying being pulled from the queue it seems).

 

The customer, in my case, added the flag and jobs were no longer delayed.

Please review the documentation for Changing the Database Provider.

 

There is a possibility of other clustered Databases where this could also present a similar behaviour but I have little evidence of this yet and one possible suspect. Please do ensure the DB_JDBC_URL in the FME Server Config file(s) are properly set for the clustered database you are using.

 

If you have any questions don't hesitate to post a new question or reach out to Safe Software Support.


steveatsafe
Safer
Forum|alt.badge.img+12
involver wrote:

I have also noticed that if I go to Licensing & Engines -> Engine page, after a few seconds I get this error.

0684Q00000ArLmWQAV.png

I did what was explained if you hit the link, but it did not solve the problem. I have a feeling that this may be connected to the issue I am having.

 

Another thing that could be happening is if the Core can’t see the Distributed Engine (port 7500) this REST API error will get returned when visiting the Engine Management page.  The core attempts to interrogate the engine manager on the distributed engine host to get the engine counts.  This is important to resolve by ensuring the port is open on the distributed engine host.  A good tool to use in this case is Powershell and test-netconnection command as this will ensure connectivity is possible.  

Open Powershell, and run this from the FME Flow Core system to the distributed engine host

i.e. test-netconnection -computername GISSERVER01DEV -port 7500

If you are experiencing such issues, one way, there’s a good chance the distributed engine host is having some connectivity issues, the other way as well… and this could lead to the FMEFlowJobSubmitter misbehaving.  This transformer will establish connectivity from one Job (parent) to another (child), waiting for results.  The issue in this post seems to suggest port issues, and is likely on the ephemeral ports.

Please review the FME Flow Ports documentation to ensure all necessary ports are open.  

It is important to know that all nodes have proper connectivity via ports that FME Flow will use.  There are always ephemeral ports in play for FME Flow environment, a regular part of network connectivity between systems, and if you have restrictions in your environment (not so common), please ensure to tweak the FME Flow configuration and update the port pool range.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings