Question

Getting occasional "Extremely low initial memory" warnings, how to troubleshoot?

  • 21 January 2022
  • 7 replies
  • 44 views
V
Badge

Hi,

 

I see these warnings in some of our logs (FME Server 2021):

 

FME Configuration: Extremely low initial memory conditions detected. To improve stability and performance please increase the memory available to FME.

 

It seems to be random, not correlated to any particular jobs. However, when it happens, it seems to affect two to four jobs in a row - I don't have many jobs yet (recent installation), so this might be a coincidence. On the bright side, it doesn't seem to prevent the jobs from finishing.

 

After reading a few articles, I see a few points I suspect I could look into:

 

  • FME Server configuration - specifically, MAX_TRANSACTION_RESULT_SUCCESSES and MAX_TRANSACTION_RESULT_FAILURES. We have the default values at this moment (100 and 10).
  • Memory parameters - in the examples I see "Start freeing memory when the process exceeds 71.99 GB". In our system we see much lower values, e.g., "Start freeing memory when the process exceeds 2.83 GB or address space exceeds 3.41 GB".

 

I don't know how to approach this though. Any suggestions?

 

Let me add that this is a new server running Windows Server 2019. We didn't have those warnings in the old server it replaced, which was running Windows Server 2008 and FME Server 2018.

 

Thank you,

 

Vini

 

 

7 replies

david_r
Rank
Userlevel 4

Realistically (and in my experience), there are only two things that really fixes the issue:

  1. Add more RAM to the server
  2. Free up RAM on the server by disabling/stopping unneeded applications and services

It's always possible to play around with the memory parameters and it may help a little in the short term, but I wouldn't expect miracles. E.g. you can increase disk swap space, but it's going to slow down everything on the server. And you can allow FME to consume more memory, but it's going to be to the detriment of the other systems on the server.

That said, there is a third fix, although it's not always feasible: rewrite your workspace to consume less memory. Typically this entails reworking blocking transformers and reducing how much data is read into FME, see https://community.safe.com/s/article/blocking-transformers-and-the-flow-of-features and https://community.safe.com/s/article/tutorial-let-the-database-do-the-work

david_r
Rank
Userlevel 4

Of course, also make sure that you're using the 64-bit version of FME Server, but I assume you already do...

V
Badge

Well... no, it's 32-bit, because the 64-bit is not compatible with other software we have (an upgrade is on the way, but it will take another year or so). But again, this hasn't changed from the old server to the new (although the version has).

 

The server has 32GB of RAM, ~10GB of which are constantly in use (for all processes, not only FME Server). It looks to me there's plenty of resources left. I haven't monitored anything though, it could be that at the time the job started, something else was taking up lots of memory.

 

About those "Start freeing memory..." messages, how are those numbers determined? I'd like to make sure they make sense, they look very low compared to the examples I see in the documentation, or compared to the server resources - even though they were the same in the old server.

 

Thanks,

 

Vini

david_r
Rank
Userlevel 4

Well... no, it's 32-bit, because the 64-bit is not compatible with other software we have (an upgrade is on the way, but it will take another year or so). But again, this hasn't changed from the old server to the new (although the version has).

 

The server has 32GB of RAM, ~10GB of which are constantly in use (for all processes, not only FME Server). It looks to me there's plenty of resources left. I haven't monitored anything though, it could be that at the time the job started, something else was taking up lots of memory.

 

About those "Start freeing memory..." messages, how are those numbers determined? I'd like to make sure they make sense, they look very low compared to the examples I see in the documentation, or compared to the server resources - even though they were the same in the old server.

 

Thanks,

 

Vini

Well, there's your problem. The 32-bit architecture does not allow using more than ~3.5 GB RAM per process (see e.g. https://techterms.com/help/difference_between_32-bit_and_64-bit_systems), so no matter how much RAM you have installed, that is going to be the upper limit of memory that can be used per FME engine.

V
Badge

OK, makes sense. Nothing to do about that, then. I mean, until we upgrade everything else.

 

So back to the original question, about that warning, "Extremely low initial memory...", is it something we can ignore?

 

Thanks,

 

Vini

david_r
Rank
Userlevel 4

OK, makes sense. Nothing to do about that, then. I mean, until we upgrade everything else.

 

So back to the original question, about that warning, "Extremely low initial memory...", is it something we can ignore?

 

Thanks,

 

Vini

I cannot tell what practical consequences this message may have, you'll have to contact Safe support for that.

But what I can tell based on my own experience, is that the workspace will likely run an order of magnitude slower, since the system will have to constantly swap memory in and out of the disk swap space.

V
Badge

Sounds good, I'll contact them. Thanks for the replies, I appreciate it.

 

Vini

Reply


Community Stats

28,741
Posts
109,578
Replies
36,581
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.