Skip to main content
  • EN

Hello,

Some time ago I had troubles with an unwanted outcome of my AttributeManager, see here:

https://knowledge.safe.com/questions/107194/what-is-causing-a-difference-in-attribute-handling.html

 

Now I have a similar issue. I have a table with 224 records, but for 3 records the outcome of the AttributeManger is not what I want and what I would expect.

It goes wrong for these three records:

This table goes into an AttributeManager where I multiply the values in the 'top' column with 1000.

I use this statement: @int(@Value(top)*1000)

 

The outcome is as follows:

Does anyone know why it does not result in these values:

LocDepth_IDtopB49F0423-8.118110B50D0035-2.012010B50D0036-4.024020

 

And why it goes wrong for only 3 out of 224 records?

 

Thanks in advance!

Eva

I suspect it's a floating point issue as described here

https://knowledge.safe.com/questions/108749/-16.html

8.11 * 1000 is actually 8109.999999999999 so when you then use int on it you get 8109 instead of the expected 8110

If all your inputs only have two decimal places you could remove the decimal point and add a 0 on the end with the text editor.

Or you could try rounding before the int function, although that might introduce it's own problems elsewhere

@int(@round(@Value(number)*1000))

I suspect it's a floating point issue as described here

https://knowledge.safe.com/questions/108749/-16.html

8.11 * 1000 is actually 8109.999999999999 so when you then use int on it you get 8109 instead of the expected 8110

If all your inputs only have two decimal places you could remove the decimal point and add a 0 on the end with the text editor.

Or you could try rounding before the int function, although that might introduce it's own problems elsewhere

@int(@round(@Value(number)*1000))

If you rounded to (say) 7 decimal places, then it shouldn't introduce other problems, should it? You'd get rid of the excess decimals by rounding up (or down) but wouldn't affect any other numbers. 

So:

@int(@round(@value(number)*1000),7)

Basically round off to greater than your own precision.

Thank your for your replies. Both work!

I will use @int(@round(@value(number)*1000),7), so hopefully it will not introduce problems elsewhere.

 

 

Reply


Community Stats

31,818
Posts
120,841
Replies
39,446
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.