RasterConvolver incorrect standard deviation calculations

  • 2 April 2024
  • 3 replies

Badge +1

Hi there :), 

I’ve been using RasterConvolver in FME Form 2023.0.1 to calculate the standard deviation with a round kernel, so some weights were equal to 0. The results were always incorrect, so to understand what’s happening I simplified the problem and used a raster with all values = 1 and one weight in kernel (3x3) = 0.

From what I understand, the formulas used in RasterConvolver for calculating the weighted mean and weighted SD are a bit off. Here is what I deducted:

The weighted mean is divided by a number of values (kernel cells) instead of by the sum of them. 

The weighted SD: in the sum equation the weight is multiplied by a cell value instead of by a square of the difference between a cell value and a weighted mean AND the sum equation is divided by a number of values instead of a sum of weights.

This may seem pretty convoluted (pun intended :)), so I have attached a pdf with a visual explanation. I additionally attach a fmwt file with two examples of RasterConvolver misscalculations. One with raster with only 1s and second with my data.


  • Firstly to FME folks: is there a possibility of changing/patching those equations in the RasterConvolver Transformer?
  • And to everyone: Is there any way you recommend to convolve raster without RasterConvolver? Ideas that came to me are:
    • changing raster to vector, saving kernel size neighbourhood values in lists and later doing calculations using list values in attributes
    • dividing the initial raster into a number of overlapping kernel size rasters for each cell, doing calculations on each of them separately and merging them afterwards

Both ideas seem a bit clumsy as well as time and RAM absorbing. I’m more of a beginner in FME, so I’ll welcome any advice you can give me in this matter :)

3 replies

Userlevel 6
Badge +32

When you suspect something is wrong with FME, it’s probably wise to submit a case at Safe Software:

If it is a bug, they will prioritize and fix it in a newer version.

Badge +1

Thank you for letting me know. I'll do that :)

Userlevel 4
Badge +25

Hi @agajo 

It’s definitely worth filing a support request if you haven’t done that already. I’ve chatted with our raster developers and there are two things that we think are incorrect.

  1. The documentation says that zero weights don’t participate in the calculation, but that’s not what you and I are seeing at all.
  2. We’re just calculating the regular standard deviation but, as you said, a weighted standard deviation should be calculated differently. We may want to add a parameter to allow users to choose which approach to take.

Anyway, this is to say that I think we can certainly do better here. So definitely file a case with support and they will be able to file some enhancement requests and - I hope - provide some workarounds for you.