Dynamic raster recoding

I once encountered an old process that did this using a looping custom transformer. It worked, but only as long as there wasn’t any overlap in the old and new pixel values. For example, if 0 was mapped to 1, and 1 was mapped to 2, the result was that all pixel values of either 0 or 1 would become 2. Not ideal.

I ended up doing the cell value replacements in a PythonCaller so that I could do all the replacements in a single pass. It’s also quite a bit faster this way.

Here’s the (very old) code in case it’s helpful to you, hopefully it still works in Python 3:

import traceback
from fmeobjects import *

# Dictionary containing before and after values
# for pixel replacement. Modify as needed.
VALUES_TO_REPLACE = {
    0: 0,
    1: 2,
    2: 11,
    3: 21,
    4: 31,
    5: 41,
    6: 51,
    7: 61,
    8: 71,
    9: 81,
    10: 91,
    11: 101,
}


def get_interpretation_class(interpretation):
    if interpretation == FME_INTERPRETATION_REAL64:
        return FMEReal64Tile
    elif interpretation == FME_INTERPRETATION_REAL32:
        return FMEReal32Tile
    elif interpretation == FME_INTERPRETATION_UINT64:
        return FMEUInt64Tile
    elif interpretation == FME_INTERPRETATION_INT64:
        return FMEInt64Tile
    elif interpretation == FME_INTERPRETATION_UINT32:
        return FMEUInt32Tile
    elif interpretation == FME_INTERPRETATION_INT32:
        return FMEInt32Tile
    elif interpretation == FME_INTERPRETATION_UINT16:
        return FMEUInt16Tile
    elif interpretation == FME_INTERPRETATION_INT16:
        return FMEInt16Tile
    elif interpretation == FME_INTERPRETATION_UINT8:
        return FMEUInt8Tile
    elif interpretation == FME_INTERPRETATION_INT8:
        return FMEInt8Tile
    elif interpretation == FME_INTERPRETATION_GRAY8:
        return FMEGray8Tile
    elif interpretation == FME_INTERPRETATION_GRAY16:
        return FMEGray16Tile
    elif interpretation == FME_INTERPRETATION_RED8:
        return FMERed8Tile
    elif interpretation == FME_INTERPRETATION_RED16:
        return FMERed16Tile
    elif interpretation == FME_INTERPRETATION_GREEN8:
        return FMEGreen8Tile
    elif interpretation == FME_INTERPRETATION_GREEN16:
        return FMEGreen16Tile
    elif interpretation == FME_INTERPRETATION_BLUE8:
        return FMEBlue8Tile
    elif interpretation == FME_INTERPRETATION_BLUE16:
        return FMEBlue16Tile
    elif interpretation == FME_INTERPRETATION_ALPHA8:
        return FMEAlpha8Tile
    elif interpretation == FME_INTERPRETATION_ALPHA16:
        return FMEAlpha16Tile
    else:
        return None


class DynamicBandTilePopulator(FMEBandTilePopulator):

    def __init__(self, dataArray, interpretation):
        self.dataArray = dataArray
        self.interpretation = interpretation
        self.tiletype = get_interpretation_class(interpretation)

    def clone(self):
        return DynamicBandTilePopulator(self.dataArray, self.interpretation)

    def getTile(self, startRow, startCol, tile):
        numRows, numCols = len(self.dataArray), len(self.dataArray[0])
        newTile = self.tiletype(numRows, numCols)
        newTile.setData(self.dataArray)
        return newTile

    def setDeleteSourceOnDestroy(self, deleteFlag):
        pass

    def setOutputSize(self, rows, cols):
        return (rows, cols)


class ReplaceCellValues():

    def __init__(self):
        self.feature_count = 0
        self.log = FMELogFile()

    def _log(self, message, severity=FME_INFORM):
        """
        Sends message to the FME log file and console window.
        Severity must be one of FME_INFORM, FME_WARN, FME_ERROR,
        FME_FATAL, FME_STATISTIC, or FME_STATUSREPORT.
        """
        self.log.logMessageString(message, severity)

    def input(self, feature):
        """Called once for each raster feature."""
        try:
            self.feature_count += 1

            raster = feature.getGeometry()
            if isinstance(raster, FMERaster):
                raster_properties = raster.getProperties()
                row_count = raster_properties.getNumRows()
                column_count = raster_properties.getNumCols()
                num_bands = raster.getNumBands()
                new_raster = FMERaster(raster_properties)
                for band_no in range(num_bands):
                    band = raster.getBand(band_no)
                    nodata_value = band.getNodataValue()
                    band_properties = band.getProperties()

                    # Force band tile to same size as input raster, this makes
                    # everything easier since we don't need to implement
                    # tile-based processing for such small rasters
                    band_properties.setNumTileCols(column_count)
                    band_properties.setNumTileRows(row_count)

                    interpretation_class = get_interpretation_class(
                        band_properties.getInterpretation()
                    )
                    tile = band.getTile(
                        0, 0, interpretation_class(row_count, column_count)
                    )
                    raster_data = tile.getData()
                    for row in range(row_count):
                        for column in range(column_count):
                            cell_value = raster_data[row][column]
                            if cell_value == nodata_value:
                                continue  # Do not replace NODATA values
                            if cell_value in VALUES_TO_REPLACE.keys():
                                raster_data[row][column] = VALUES_TO_REPLACE[cell_value]
                    band_properties.setTileType(FME_TILE_TYPE_FIXED)
                    band_tile_populator = DynamicBandTilePopulator(
                        raster_data, band_properties.getInterpretation()
                    )
                    new_band = FMEBand(
                        band_tile_populator, raster_properties, band_properties
                    )
                    new_raster.appendBand(new_band)
                feature.setGeometry(new_raster)
            else:
                feature.setAttribute("_error", "The geometry is not a raster.")

            self.pyoutput(feature)

        except:
            self._log("=" * 78, FME_ERROR)
            self._log(
                "PythonCaller(%s) exception at feature number %d:"
                % (self.__class__.__name__, self.feature_count),
                FME_ERROR,
            )
            self._log("-" * 78, FME_ERROR)
            for line in traceback.format_exc().splitlines():
                self._log(line, FME_ERROR)
            self._log("=" * 78, FME_ERROR)
            if feature:
                self.log.logFeature(feature, FME_ERROR)
            raise FMEException(
                "An error occurred in PythonCaller '%s'. "
                + "See log for details." % self.__class__.__name__
            )

    def close(self):
        """Called once after the last feature has been processed."""
        self._log(
            "PythonCaller(%s): %d feature(s) processed"
            % (self.__class__.__name__, self.feature_count),
            FME_STATISTIC,
        )

You’ll have to set the contents of the dictionary VALUES_TO_REPLACE yourself, e.g. based on an attribute list.

The routine has only ever been used on small-ish rasters, and for the sake of simplicity the code does not implement tile-based processing: it simply uses a single tile the same size as the entire raster. If you have very big rasters it might be that you have to extend the code to use smaller tiles and process them separately. It shouldn’t be too hard, I believe.

I also found an old (FME 2012!) looping custom transformer that iterated through the list and replaced the values one at a time, the domain issue was addressed by adding 1000 to the output values and then subtracting 1000 once all the recoding was done, but it was quite inefficient.


I had a quick look last night at replaceCellValuesMultipleRanges  in the python api but kept getting a type error when I tried to implement it.  My python skills are mediocre so I could be missing something obvious.

Message Type: fme::internal::_v0::py::Exception
Python Exception <TypeError>: descriptor 'replaceCellValuesMultipleRanges' for 'fmeobjects.FMERasterTools' objects doesn't apply to a 'list' object
 

import fme
from fme import BaseTransformer
import fmeobjects


class FeatureProcessor(BaseTransformer):
    def __init__(self):
        pass

    def input(self, feature: fmeobjects.FMEFeature):
        inList = feature.getAttribute('inList{}')
        outList = feature.getAttribute('outList{}')

        raster = feature.getGeometry()
        band = raster.getBand(0)

        recode = fmeobjects.FMERasterTools.replaceCellValuesMultipleRanges(inList,inList,outList,False,band)

        self.pyoutput(feature, output_tag="PYOUTPUT")

    def close(self):
        pass

Ahh, I missed the () in FMERasterTools.  I just copied the reference straight from the documentation :(.

I didn’t get to the exporting the raster portion of the code because replacing the cell values wasn’t working.

As an FYI there is also fmeobjects.FMERasterTools.replaceCellValues  if you only want to work with a single range.

And yes i think they are relatively new methods.