Main question
Can anyone advise me about the installation of the GDAL Python module and its implications on FME's Python interpretations? Can anyone share their experiences in using a different Python module within FME other than FME's Python interpreter or ArcPy?
Background info
Currently, it is not directly supported to calculate percentile values in the *StatisticsCalculator transformers of FME (e.g. also not in the RasterStatisticsCalculator). See also this Idea on the FME community.
In order to still do so, I thought about using a Python script/module to perform this step. Specifically, I would like to extract the 95% percentile of pixel values in a GeoTIFF (those pixel values represent absolute height values), for portions of the GeoTIFF that overlap/intersect with 2D polygon features of building extents.
Python modules likely suitable for the task of calculating 'zonal statistics over a raster'
One might use the 'Zonal Statistics As Table (Raster Analysis)' from ArcGIS Pro, which relies on the ArcPy Python module, but this requires Esri licenses, and these are quite scarce in our organization (and by floating license not always available).
Then I also found out about the rasterstats Python module, which doesn't rely on an Esri license. So I wanted to give this a try. However, this module seems to require the GDAL module/library of Python, and I'm not sure if it is recommended to install/link that module/library with FME's Python interpreter. See also below
Installing the rasterstats Python module and requirement on GDAL
To install a Python packages/modules for FME Desktop, I had a look this documentation page of Safe for this. I used the following command for this in command prompt:
fme.exe python -m pip install rasterstats --target "C:\Users\<USERNAME>\OneDrive - <ORGANIZATION>\FME\Plugins\Python"
That resulted in an error:
A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable
Now at the rasterstats page there is some info on how to install it here, particularly for windows it is described to follow the rasterio installation, in which under the 'Source Distributions' section, subsection 'Windows', it is described how/where to download a binary distribution of GDAL (here), and how to install it.
Only, when I look a bit further, I found this tread on the FME community. I am a bit thrown off by david_r comment "I do seem to remember, however, that getting the gdal Python bindings to work properly on Windows was a royal pain, but hopefully it's gotten better by now.", and that he described/hinted to using a stand-alone Python interpreter, whereas I would like to the Python interpreters of FME.
I'm not sure however if doing so would interfere with how FME handles geometric operations... Therefore, before doing so I wanted to ask for some advice on this...
Some closing remarks/info
- The underlying goal/task is actually identical to this thread on the FME community (there they try to achieve the same with an Rcaller).
- I actually managed to make this work using FME only, by using a RasterCellCoercer to get individual points for each raster, creating a list of pixel values for each building, sorting the list using a ListSorter, extracting the appropriate list element(s) for the percentile value, and then setting the percentile value (using manual interpolation between two list element values).
- As such, I'm not looking for advice on how to do this in FME - since that approach is functional already.
- You might wonder "Then why still try the Python approach?". The scope of this question reaches further than just this specific Rasterstats functionality: it is part of an effort to experiment with using Python more inside of FME workspaces, so that we can extend FME with Python's power whenever necessary.