Hi,
This question is linked to a previous question I posted on the community, about the number accuracy of the Esri ASCII GRID writer of FME.
Please see that question for more details and screenshot, but a quick recap of it; when I created a dummy raster with Real32 interpretation, in which an original pixel value would be '0.0250313',
the rounded pixel value would become '0.025',
but the Esri GRID File seems to convert to Real64 interpretation and stores the value as '0.02503128908574581',
whereas I would like the rounded value '0.025' to be written to the file. I noticed that this could save up to 50% in the file size. I suppose that this item is related to the Floating Point Precision and String Formatting in FME.
Now I have been investigating some options for an efficient workaround to this question. I.e. to be able to create Esri ASCII Grid files that don't write out pixel values with 16 decimal values. I came up with two methods, but I think they are both not optimal;
- Write the Esri ASCII GRID file as regular, read in the result, update the data lines, and create a new smaller version of the file
- Fetch the relevant Raster properties, coerce the raster to a PointCloud and then to a MultiPoint, Fetch the (Z) coordinates as a list, and manipulate the list into rows of pixel values.
I added a screenshot of the steps and the workspace itself to this case for who is interested.
Upon doing so I came across a couple of questions.
1. Do people maybe have some other tips/suggestions for me on other more efficient implementations?
That is because both of my methods have some limitations. The first option ends up creating individual features for each pixel, which results in a lot of features and thereby poor performance for large rasters.
The second method does seem to be already a bit more efficient, but it seems to use quite much RAM/memory, and FME also warns me about this when it's creating the list of coordinates for the MultiPoints (Z value representing the pixel values).
Another sideremark in my implementation of method 2.
2. It seemed to me that the RasterBandPropertyExtractor returns a list attribute in which the list attribute '_band{0}.band_num_tile_rows' seems incorrect to me (It seems to always return a value of '1').
Luckily the RasterPropertyExtractor does seem to return the correct number of rows and columns for the raster. So could it be that this fixed value of 1 for the number of rows is a bug in the RasterBandPropertyExtractor transformer?
Then at last I think that the most of my 2nd method could be simplified by using Python, using the Python FME Objects API for Raster Manipulation.
Specifically I think that the getData() function of one of the FMETile* classes (in particular the FMEReal32Tile class), seems to return the raster data almost directly in the structure I would need it to create the updated Esri ASCII Grid.
Unfortunately I'm not so experienced with Python. I had a look online and I found two items with examples of how the Python FME Objects API for Raster Manipulation was used. The example from takashi, and another one from a blogpost on another forum.
While I was able to replicate these examples (I also embedded these in my attached workspace), unfortunately I'm a bit too new with Python and classes to succesfully invoke the getData() Python function for my specific raster.
3. Is there anyone in the community that can maybe help me out with a working PythonCaller script to succesfully invoke the getData() Python function to get the Real32 bit value Raster data values as a nested list?
Any feedback/help would be greatly appreciated!
Kind regards,
Thijs
