• EN
Question

"Bad Allocation" error using MapnikRasterizer?

  • 3 December 2015
  • 4 replies
  • 8 views
B

Hi guys,

Has anybody else run into an issue using the MapnikRasterizer where the translation terminates unexpectedly with the error "Bad Allocation"? My first thought was RAM, but I'm running 64bit FME 2015.1.2 with plenty of available disk space and physical RAM (10+GB free at the time of the error) so I doubt it is that.

I am trying to rasterise Ordnance Survey Mastermap data using the EPSG:3857 Bing Maps grid. The error seems to occur randomly, and frustratingly kills the whole process so no images are written to the file system (although multiple tiles do pass out of the MapnikRasterizer before the issue occurs).

Any help greatly appreciated!

4 replies

fmelizard
Rank
Userlevel 4
Badge +13

I believe this is a known issue with the MapnikRasterizer and we have an existing problem report - PR#54204 - documenting it. Apparently the determining factor is the amount of virtual memory the process can use and so a potential workaround would be to rasterize the image in tiles and mosaic them for output, instead of making one large image directly. I have added this thread to the problem report in case there is any followup.

Regards,

Robyn Rennie

fmelizard
Rank
Userlevel 4
Badge +13

After @RobynAtSafe escalated this at Safe, the team brainstormed a few ideas as workarounds until we can figure out a better long term solution.

We're guessing that you must be using a Group on your MapnikRasterizer to gather features for each tile. And since it is coming from MasterMap, we're guessing there are lots of features and lots of XML-birthed attributes on each one. So we'd recommend putting an AttributeKeeper before the MapnikRasterizer on each input port and keeping only the attributes you reference in the MapNikRasterizer. That will greatly reduce the memory overhead and should free up a bunch. Additionally, we feel this may be a great place to engage the Parallel Processing. If you were using a Group, then each group would get processed in its own process and as such have a much roomier memory situation.

Give those a spin and let us know if it helps.

B

Thank you both, your assumptions are spot-on Dale so I will try the suggestions and report back!

steveatsafe
Rank
Badge +11

Thank you both, your assumptions are spot-on Dale so I will try the suggestions and report back!

@bhawk Hope you are well. It's been a while since this topic was created... but thought I'd see about an update. The PR on this that Robyn reported is still open. Did you happen to resolve your issue with Dale's recommendations?
SteveatSafe

Reply


Community Stats

28,804
Posts
109,800
Replies
36,621
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.