Attribute rounder - 0.145

Hi,

 

 

rounding is surprisingly complicated. Your observation is due to inaccuracies when translating between decimal and binary floating point numbers.

 

 

See the note here for more info: https://docs.python.org/2/library/functions.html#round

 

 

This is of course not specific to FME and it is a pretty well known problem within computer science, see e.g. http://floating-point-gui.de/

 

 

For your case, you can try to work around it using an ExpressionEvaluator with a "manual" rounding:

 

 

@round(@Value(input)*100)/100.0

 

 

This gives me a result of 0.15.

 

 

David

To add a bit to the specific case of 0.145:

 

 

0.145 (decimal, what you see) = 00111110 00010100 01111010 11100001 (binary, what the computer stores)

 

 

When you convert the binary representation back to decimal, however, you will see that the conversion introduced a slight "rounding error", since not all decimal numbers can be accurately represented in binary. In this case, we get 0.14499999582767487

 

 

Now, if you round 0.14499999582767487 to the nearest two decimals you will get 0.14.

 

 

So 0.14 is the correct answer to the computer, although maybe not the one the user expected :-)

 

 

Hope this makes sense.

 

 

David