Skip to main content
  • EN
Hi,

 

 

I am having problem with running fme workbench, actually, it terminates when using Clipper.

 

Here is a part from log:

 

 

Finished clipping 117400 / 4960440 clippees against all clippers Finished clipping 120800 / 4960440 clippees against all clippers Performing low-level intersection at phase #1... 100% done Insufficient memory available -- error code was 2 - please read the FME Help section 'Adjusting Memory Resources' for workarounds. Out of memory.  Program terminating.  Please read the FME Help section 'Adjusting Memory Resources' for workarounds.

 

I read about adjusting memory and it is recommended to try use 3GB Switch, and there is a note: "Note: The /3GB switch is not required for all, or even most, users of FME. The /3GB switch will only benefit those who require translations that will not currently run with 2GB of addressable memory."

 

The OS is Windows Server 2008 R2 Enterprise,4GB RAM,64bit.

 

I thought a little bit about doing some changes within workbench, but I cannot avoid CLipper transformer and amount of data that already have been reduced.

 

Any tips?

 

Thank you.

 

 
Hi,

 

 

this can be a bit tricky, but here are some alternatives:
  • Use FME 64-bit on a 64-bit OS, if possible
  • If you have a lot of overlapping clippers, try dissolving them first to reduce the number
  • Pre-process your input to distribute your objects over several clippers / workbenches. You can do this by first running all your objects through a BoundingBoxAccumulator, then using the Tiler to split the bbox into a suitable number of polygons that you give a number from 1..n. Use a SpatialFilter with INTERSECT to give each clippee and clipper their corresponding tile number, then use this as a Group By in the Clipper, or spread it out over several Clippers depending on their tile number (e.g. AttributeFilter).
  • Use the ModuloCounter approach used by SigTill here (link to download workspace). It should also apply to a clipping operation.
I'm sure there are other methods as well.

 

 

David
Ok David. First suggestion is not possible to try because of lack of format, second suggestion I tried, but still does not help. Working with third now :)

 

 

I will write later about results.

 

Thank you.
Hi,

 

have you tried the paraller processing otion in the clipper?

 

 

This webinar might be helpful to you:

 

 

http://youtu.be/ecR74ozMQdM

 

The Clippers First option will - if you aren't using it already - make a HUGE difference to you here. Without it FME will be caching those 4960440 clippees in memory, and that will not help performance!

 

 

This article, especially the section on memory resources, may help explain.

 

 

If it's running out of memory, and you are using Clippers First - then either you must have millions of clip boundaries, or there is a problem you should contact our support team about.

Regards

 

 

Mark Ireland

 

Product Evangelist

 

Safe Software Inc
@Mark I use around 500 clippers and not using clippers first option.

 

@Itay I will try parallel processing, altough  i do not know how to group clippers because each clipper is with different attributes value.
This short vid explains parallel processing  and demonstrates how to create groups.

 

Remember that the cost of transporting the data should not exceed the amount of work than needs to be done. Also the setting of parallel processing should be done in accordance with the cpu numbers.

 

 

but,

 

 

usually what Mark suggests is spot on.

 

 

to use the option of clippers first, move (or slide with fme 2013) the clippers reader to the top of the navigator.

Reply


Community Stats

30,152
Posts
114,462
Replies
37,797
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.