Skip to main content
Question

Insufficient memory available usage

fmeuser_gc
Contributor
Forum|alt.badge.img+8

I get this error for only this scheduler job. All other jobs fine.

Insufficient memory available -- error code was 2 - please read the FME Help section 'Adjusting Memory Resources' for workarounds.

Nothing much to clean up in /temp folder. How can I make this job run?

Yash K

    10 replies

    M
    Forum|alt.badge.img+2
    • mark_f
    • January 10, 2018
    Probably would help to explain about the context of the job, server environment 32/64bit etc.

     

    mark2atsafe
    Safer
    Forum|alt.badge.img+43
    • mark2atsafeSafer
    • Safer
    • January 10, 2018
    Also, how much data is there? Sometimes this is due to really running out of memory (keep an eye on the system resources while running, to see if it is being constantly reduced) but I have also seen such an error occur because of a piece of bad data. If the dataset has only a few features it might be a corrupt feature rather than a lack of memory.

     

    fmeuser_gc
    Contributor
    Forum|alt.badge.img+8
    • fmeuser_gcContributor
    • Author
    • Contributor
    • January 10, 2018
    mark2atsafe wrote:
    Also, how much data is there? Sometimes this is due to really running out of memory (keep an eye on the system resources while running, to see if it is being constantly reduced) but I have also seen such an error occur because of a piece of bad data. If the dataset has only a few features it might be a corrupt feature rather than a lack of memory.

     

    @Mark2AtSafe - Yes system resoures (memory) is being used up almost 90% of it. There is nothing wrong with the dataset as the job is successful in other environements.

     

    Yash K
    fmeuser_gc
    Contributor
    Forum|alt.badge.img+8
    • fmeuser_gcContributor
    • Author
    • Contributor
    • January 10, 2018
    mark_f wrote:
    Probably would help to explain about the context of the job, server environment 32/64bit etc.

     

    @mark_1spatial - It runs on FME 2015.1.3.2 version, 32-bit.

     

    Yash K
    mark2atsafe
    Safer
    Forum|alt.badge.img+43
    • mark2atsafeSafer
    • Safer
    • January 10, 2018
    fmeuser_gc wrote:
    @Mark2AtSafe - Yes system resoures (memory) is being used up almost 90% of it. There is nothing wrong with the dataset as the job is successful in other environements.

     

    Fair enough. Then I think you'll have to post the workspace so we can see what it is doing. Either it's a memory leak (and we'd need to run it at Safe to figure that out) or it's just more data than that environment can handle (so if we can see the workspace we might be able to figure out how to cut back on memory usage).

     

    Either way, if it's using 90% of system memory resources then changing to 64-bit won't help. Still surprising it doesn't cache data first before failing, but I guess that can happen sometimes. What is the environment, btw? How much memory does it have installed?

     

    M
    Forum|alt.badge.img+2
    • mark_f
    • January 10, 2018

     

    Not FME Server but I saw significant improvements between FME Desktop 2015 vs 2016.
    fmeuser_gc
    Contributor
    Forum|alt.badge.img+8
    • fmeuser_gcContributor
    • Author
    • Contributor
    • January 11, 2018
    mark_f wrote:

     

    Not FME Server but I saw significant improvements between FME Desktop 2015 vs 2016.
    @mark_1spatial, we are yet to upgrade it to FME 2017

     

    Yash K
    fmeuser_gc
    Contributor
    Forum|alt.badge.img+8
    • fmeuser_gcContributor
    • Author
    • Contributor
    • January 11, 2018
    mark_f wrote:
    Probably would help to explain about the context of the job, server environment 32/64bit etc.

     

    @mark_1spatial, fyi more detailed specs

     

     

    OS : Windows 2012 R2

     

    4 CPU

     

    16 GB RAM

     

    Yash K
    david_r
    Evangelist
    • david_rEvangelist
    • January 11, 2018

    As @mark_1spatial says, it's nearly impossible to give a precise answer to such a broad question, there are so many factors that influence memory consumption, and most of them related to your data and how your workspace is authored.

    However, Safe has published a great resource with a lot of general tips and tricks here: Performance Tuning FME It's a very good starting point for narrowing down the issue.

    My number one tip: If you have a lot of data passing through blocking transformers (e.g. the FeatureMerger), try your hardest to "unblock" them by carefully manipulating the data flow and how they're configured. Done right, this can drastically reduce the memory consumption while executing your workspace. More info here: https://knowledge.safe.com/articles/38700/clearing-blocking-transformers.html

    M
    Forum|alt.badge.img+2
    • mark_f
    • January 11, 2018
    fmeuser_gc wrote:
    @Mark2AtSafe - Yes system resoures (memory) is being used up almost 90% of it. There is nothing wrong with the dataset as the job is successful in other environements.

     

     

    @fmeuser_gc you said: "There is nothing wrong with the dataset as the job is successful in other environements."

     

    Can you elaborate - other FME Server environments? Running on FME Desktop? What are the differences between these and the one where it fails? Version, Memory, CPU?

    Reply


    Helpful Members This Week

    Community Stats

    30,892
    Posts
    117,078
    Replies
    38,700
    Members

    Latest FME

    Terms & ConditionsCookie settings

    Cookie policy

    We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

     
    Cookie settings
    A product of
    Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.