Skip to main content
  • EN

I have a huge ECW-file with not much data. Seen below is a simplification. The black area is "nodata area" and the red area is the actual data (ortophotos following a road).

I want to convert this to a BigTIFF (tiled). The issue is that the resulting file is too large to be stored in memory (according to stats, 60 GB of RAM ++ is needed) so it slows down alot. By dividing this area with the RasterTiler I can get approx 20 tiles that does not need that much RAM while processing.

However I wonder if it is possible to somehow "not read" the no-data,black,transparent data (as mentioned in the 4th band) - but only only read the relevant pixels.

On VECTOR datasets when you use the Tiler - it will only send the tiles containing vector-data, and remove the "empty tiles" - this does not seem to work with rasters. A solution would be to calculate band 4 on data within that raster-tile - and if all values are transparent then skip that tile. Or preferrably it might be a faster solution to this?

I tried to use the Creator - FeatureReader and the Spatial Filter with Intersect - however it seems that the ECW format is not "spatial aware" and reads the complete ECW instead of just the wanted part.

Hi @sigtill, as far as I know, there seems to be no way to skip reading/creating tiles that contain only Nodata cells. A workaround I can think of is to use the RasterBandMinMaxExtractor to extract min/max cell values for each tile, then determine whether each tile has only Nodata cells, based on the values. However, I don't know if the workaround can contribute to improve the performance of entire processing.

Hi @sigtill, as far as I know, there seems to be no way to skip reading/creating tiles that contain only Nodata cells. A workaround I can think of is to use the RasterBandMinMaxExtractor to extract min/max cell values for each tile, then determine whether each tile has only Nodata cells, based on the values. However, I don't know if the workaround can contribute to improve the performance of entire processing.

Agree, this mirrors my own experience. Unfortunately the RasterBandMinMaxExtractor can be relatively slow, but I don't know of any other solution for checking for empty rasters.

 

Side note: It'd be very interesting to know if the new raster objects in fmeobjects can do it faster than the transformer.

Reply


Community Stats

30,185
Posts
114,572
Replies
37,848
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.