Skip to main content
I have an attribute rounder which is rounding a figure to 2 decimal places with Round-off Direction set to nearest but when using 0.145 as an input is returning 0.14

 

 

1.145 as an input returns 1.15

 

 

Any ideas why, am I missing something?

 

 

FME 2014 SP2 (20140623 - Build 14339 - WIN32)
Hi,

 

 

rounding is surprisingly complicated. Your observation is due to inaccuracies when translating between decimal and binary floating point numbers.

 

 

See the note here for more info: https://docs.python.org/2/library/functions.html#round

 

 

This is of course not specific to FME and it is a pretty well known problem within computer science, see e.g. http://floating-point-gui.de/

 

 

For your case, you can try to work around it using an ExpressionEvaluator with a "manual" rounding:

 

 

@round(@Value(input)*100)/100.0

 

 

This gives me a result of 0.15.

 

 

David
To add a bit to the specific case of 0.145:

 

 

0.145 (decimal, what you see) = 00111110 00010100 01111010 11100001 (binary, what the computer stores)

 

 

When you convert the binary representation back to decimal, however, you will see that the conversion introduced a slight "rounding error", since not all decimal numbers can be accurately represented in binary. In this case, we get 0.14499999582767487

 

 

Now, if you round 0.14499999582767487 to the nearest two decimals you will get 0.14.

 

 

So 0.14 is the correct answer to the computer, although maybe not the one the user expected :-)

 

 

Hope this makes sense.

 

 

David

Ah, my favourite number! This 'trick' doesn't work anymore, at least for 0.145 specifically, it returns 0.14.

@round(@Value(input)*100)/100.0

The following will return 0.15.

(@round((@round((@Value(Value)*1000))/10)))/100

 I know there were some changes with rounding with python 3, not sure if it could be related


Reply