Question

Hi I'm currently reprojecting some 16bit Geotiffs, they are 3gb each 4 band, the writing out is incredibly slow at 40mins a tile.


My workspace is Geotiff reader to raster mosaicker to reprojector to clipper which clips it to a grid then geotiff writer. 

 

How can I speed this workspace up ?


4 replies

Badge +11

Hi @ryan92k​  can help you https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/workspacerunner.htm

Where would i place it in the workspace?

Userlevel 5
Badge +25

I don't think a WorkspaceRunner is going to help as you do need to mosaick all the tiles first.

 

One thing to doublecheck first is whether you're running the workspace with feature caching on. If that's the case you're getting a big performance hit. It's great for workspace development, not for production and especially not with these volumes of data.

 

Couple of other things to check/consider

  • Are the images stored on a network drive? Are they being written to a network drive? That might slow things down so copying them to your local computer would be a way to speed things up.
  • How much ram do you have? At 3Gb each you don't need a whole lot of them to fill up the ram and requiring FME to swap to temp files.
  • Is your FME Temp directory on a fast SSD?
  • Are there ways to process the data in smaller chunks?
  • Do you need them to be 16 bit, 4 band? If you can get away with dropping to 8 bit 3 band you'll save a lot

If I would have to do something like this I would seriously consider spinning up an Enterprise FME Cloud instance. It has 192 GB RAM and costs $10/hour, worth a try.

Userlevel 5
Badge +29

I'd consider re arranging your workspace. Do the clip first, then the mosaic, then the reprojector. Basically try to make the datasets as small as possible in the first instance, then do the processing on them. Your current process is making one massive file and then processing it

Reply