1.145 as an input returns 1.15
Any ideas why, am I missing something?
FME 2014 SP2 (20140623 - Build 14339 - WIN32)
1.145 as an input returns 1.15
Any ideas why, am I missing something?
FME 2014 SP2 (20140623 - Build 14339 - WIN32)
rounding is surprisingly complicated. Your observation is due to inaccuracies when translating between decimal and binary floating point numbers.
See the note here for more info: https://docs.python.org/2/library/functions.html#round
This is of course not specific to FME and it is a pretty well known problem within computer science, see e.g. http://floating-point-gui.de/
For your case, you can try to work around it using an ExpressionEvaluator with a "manual" rounding:
@round(@Value(input)*100)/100.0
This gives me a result of 0.15.
David
0.145 (decimal, what you see) = 00111110 00010100 01111010 11100001 (binary, what the computer stores)
When you convert the binary representation back to decimal, however, you will see that the conversion introduced a slight "rounding error", since not all decimal numbers can be accurately represented in binary. In this case, we get 0.14499999582767487
Now, if you round 0.14499999582767487 to the nearest two decimals you will get 0.14.
So 0.14 is the correct answer to the computer, although maybe not the one the user expected :-)
Hope this makes sense.
David
Ah, my favourite number! This 'trick' doesn't work anymore, at least for 0.145 specifically, it returns 0.14.
@round(@Value(input)*100)/100.0
The following will return 0.15.
(@round((@round((@Value(Value)*1000))/10)))/100
I know there were some changes with rounding with python 3, not sure if it could be related