Skip to main content
  • EN

I am running FME Server 2021.0.3, Build 21326. 36 GB Memory.

I ran 'systeminfo' in a cmd window:

Total Physical Memory:    36'863 MB

Available Physical Memory: 13'356 MB

Virtual Memory: Max Size: 42'239 MB

Virtual Memory: Available: 8'813 MB

Virtual Memory: In Use:   33'426 MB

 

I've noticed several warnings related to memory and occasionally jobs fail with this error and want to ensure I have things configured optimally.

  • FME Configuration: Extremely low initial memory conditions detected. To improve stability and performance please increase the memory available to FME. Available memory can be increased by adding physical RAM, increasing swap space, or closing other applications.
  • 2021-7-12 14:17:28 | Failed to free sufficient memory to reach the process usage limit. To improve stability and performance please increase the memory available to FME. Available memory can be increased by adding physical RAM, increasing swap space, or closing other applications.
  • Some crashes with this message: «ResourceManager: Optimizing Memory Usage. Please wait...» 

 

I found this article which refers to "Controlling the Automatic Restart of FME Engines". How do I work out what best to set for these parameters?

MAX_TRANSACTION_RESULT_SUCCESSES=

MAX_TRANSACTION_RESULT_FAILURES=

 

We have FME_ENGINE_MEMORY_REDLINE set to 0.7 - is that still useful in this release?

There may be a knowledge base article on this subject, but I didn't find anything specific, so any help would be appreciated.

I've encountered similar memory issues with FME server. It came down to the fact that the machine FME Server was running on was virtualised. On the host, the FME VM was configured to use up to ~20gb of RAM but none of it was specifically reserved for the VM, in other words, the host was effectivly over allocating its available RAM resources. Once the VM was reconfigured to have dedicate RAM resources we noticed that error was less frequent.

Similar to @hkingsbury​ I've seen this on VM's that are set up to dynamically increase the RAM as needed, but FME doesn't know this when the job is started so thinks it has to work with a limited amount of RAM.

Check also that you have plenty of space available in the machine (not just the FME temp location). Virtual memory uses hard disk space so if you are running low on space you can end up in a failure because there is not enough space.

 

I would just leave the redline value alone, leave it as the default.

 

Another thing to note is that by default FME Server is also set up to automatically retry jobs that crash (not fail), https://docs.safe.com/fme/html/FME_Server_Documentation/AdminGuide/Job_Recovery.htm When you have a job that crashes because of memory this can clog up the system, especially if the job is a long one, the job will retry and continue to retry. This can quickly lead to a tonne of temporary files that don't get cleaned up - you could consider setting this to 1 or 0 if you find that this is causing frequent headaches.

 

For MAX_TRANSACTION_RESULT_SUCCESSES I think you'd just need to experiment for a bit if you want to change them. The benefit of having them not restart more often is performance, if you have a lot of small jobs the process of restarting the engine every time will slow the process. If you do not have such a large load then this number can be pretty low as it shouldn't have too much effect on performance.

 

Sometimes it helps to also just restart the machine, this is not a great long term solution, however, it can bring short term benefits when it comes to memory.

 

Reply


Community Stats

30,351
Posts
115,341
Replies
38,080
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.