Skip to main content

Hello everyone,

I am having random timeouts on FMEServer when i am trying to connect to a PostgreSQL database (using FeatureReader/FeatureWriter) or when i am trying to realize a simple get/post with HTTPCaller. The timeouts seems to happen randomly, mostly when there are multiple simultaneous jobs.

The HTTPCaller timeout is partially solved by activating the option "Retry Failed Requests", but sometimes the maximum value of "Maximum Retry Attemps" (10) is not enough (i.e. the timeout is still there after 10 retries).

For the FeatureReader/FeatureWriter there is no retry option available.

A similar problem was found here It seems that i have an unstable connection to my postgres db, sometimes it will read from it and other times I get this error message: (safe.com) but there is no hints to the source or solution of this error.

Some details:

FME Server 2022 Build 22792 with 5 engines

Using Linux (RHEL 8.7) with docker

Thanks in advance

The issue in the referred question was the firewall, where only the ip of the first engine was registered. When FME server used the second engine, which was not registered, the connection started to fail.

 

I'm by no means an expert, but if feels like you hit some limit outside FME. I think I would look in the postgres log for a clue. And if that returns no issues, look in the firewall log.

 

You can always test with a dummy workspace / table, hitting it with a lot of simultaneous requests (HTTPCallers can do this) and see when it starts to stutter, but this reverse engineering often takes more time and offers no solutions, other than a Decelerator to slow down the features, which is meh.


The issue in the referred question was the firewall, where only the ip of the first engine was registered. When FME server used the second engine, which was not registered, the connection started to fail.

 

I'm by no means an expert, but if feels like you hit some limit outside FME. I think I would look in the postgres log for a clue. And if that returns no issues, look in the firewall log.

 

You can always test with a dummy workspace / table, hitting it with a lot of simultaneous requests (HTTPCallers can do this) and see when it starts to stutter, but this reverse engineering often takes more time and offers no solutions, other than a Decelerator to slow down the features, which is meh.

Thanks for the reply !

The linked issue had the same error message and referred the usage of multiple engines, so I thought that there could be a relation with the current issue. I also expected some limitations outside FME (since the FME and Postgres services are not in the same machine), but I did not find any relevant messages in system (/var/log/messages) or postgres logs (using docker-compose logs), even after launching dozens jobs non-stop.

I am also by no means an expert in the area, but i have a theory that this problem could be associated to the concurrency of connections towards outside of the docker environment, since i am getting the same timeout error for both HTTPCaller and FeaureReader\\FeatureWriter


Do you get the same random timeouts regardless of the website you're trying to fetch from with the HTTPCaller?

Also with the HTTPCaller you can turn on debug logging in FME to try and get some more info.

The HTTPCalller has the option to limit the number of concurrent requests so you could try and set that to 1 and see if it makes a difference.


Do you get the same random timeouts regardless of the website you're trying to fetch from with the HTTPCaller?

Also with the HTTPCaller you can turn on debug logging in FME to try and get some more info.

The HTTPCalller has the option to limit the number of concurrent requests so you could try and set that to 1 and see if it makes a difference.

Thanks for the tip, i'll check the debug options. I didn't check other adresses (server does not have access to internet, just some intranet services)


Reply