Skip to main content
Hi all,

 

 

We're currently processing a 1.6gb jpeg2000 file. We're applying a clipper to break it down to a 4x4 grid and then resampling using bilinear interpolation within the RasterResampler transformer.

 

 

Its setup to clip first the resample.

 

 

All files are running locally. The issue is at current estimates, per grid cell will take 40 hours.

 

 

Strangely enough, we processed a 1.2gb jpeg2000 file a while ago with same setting presumadly, and took 19 hours for the entirety of the project. Not sure whats happened.

 

 

Wanted to ask if any has tips/tricks to improve speed or recommend different transformer combinations.

 

 

Cheers,
Hi,

 

 

Maybe the RasterTiler would prove to be a better choice to clip the raster?

 

Also using the parallel processing option on the clipper can improve the preformance. see http://fmepedia.safe.com/articles/Samples_and_Demos/Parallel-Processing

 

 


If the image quality is not much of an issue, the NN option of resampling is usually faster than bilinear.

 

Hope this hepls
NN sampling  is usually faster than bilinear, but if image quality is important to you stick to bilinear.

 

Hope this helps
It's uncertain, but the RasterCheckpointer transformer may help. I'm fairly unclear about how this works (and in what scenarios) but it does sometimes help to include it between two raster operations.

 

 

So, put it between the Clipper and the RasterResampler and see if it helps. Run it on a smaller sample dataset if you have one, just to see if it helps.

 

 

Hope this helps

 

 

Mark

 

 

pPS: What I *think* it *might* do: the RasterResampler may be working on the entire 1.6gb dataset, not a tile, because raster writers work by calculating what data will be produced, and then processing the entire source. They wouldn't necessarily work on the mid-translation tiles as a vector writer would. So, by adding a CheckPointer you are forcing the tiled data to be treated as a new "source", which may be quicker]
I feel I should do a follow up to my question. I left it to run over the weekend and it took approximately 2 days to process 1 cell of 16. This used clipper -> rasterresampler(Bilinear).

 

 

I've now included the rasterCheckPointer recommended by Manitoba Mark, and we're seeing huge decrease in processing time for each cell. Like 15 MINUTES!. OMG WHAT I hear you say. Believe it.

 

 

Solution at the moment was clipper -> rasterCheckPointers  -> rasterResampler(Bilinear) -> save.

 

 

Thanks a lot!

 

 


Thanks for the tip for adding a RasterCheckpointer to workspaces.

 

 

We have a workspace that generates GeoTIFF files from a TIFF and DGN that took 3 hours to run on average, using bilinear interpolation. With the RasterCheckpointer added between the RasterResampler and RasterRotationApplier the workspace running time reduced to 1 min 20 sec.

Reply