• EN
Question

@int function is rounding up in some cases

  • 27 February 2020
  • 9 replies
  • 32 views
jakethepainter
Rank
Badge +5

FME 2019.2 desktop user. I have some dates that have fractional seconds and I want to truncate the fractional seconds to whole seconds. In the fme help for the arithmetic editor it says the @int function converts by truncation if it is in the range of a 64 bit integer, which is exactly what I want.

Most dates I'm sending in are truncated. But ones of the type 20200220015759.999000 are getting rounded up. Which, in the case of above, is an invalid date because it rounds up to 60 seconds.

Is this a bug or am I missing something?

 

9 replies

david_r
Rank
Userlevel 5

I agree that this is unexpected behavior, I was able to reproduce it as well. Even the AttributeRounder set to round down with 0 decimals gave the wrong result.

Unless someone comes up with a good workaround, consider forwarding the case to Safe support, pointing back to this thread.

ebygomm
Rank
Userlevel 1
Badge +21

I agree that this is unexpected behavior, I was able to reproduce it as well. Even the AttributeRounder set to round down with 0 decimals gave the wrong result.

Unless someone comes up with a good workaround, consider forwarding the case to Safe support, pointing back to this thread.

Isn't this a floating point issue? The following in python returns 20200220015760.0

david_r
Rank
Userlevel 5

Isn't this a floating point issue? The following in python returns 20200220015760.0

Looks like you're right:

and

ebygomm
Rank
Userlevel 1
Badge +21

Looks like you're right:

and

Someone told me about floating point issues a while back :-)

https://knowledge.safe.com/questions/3538/attribute-rounder-0145.html

david_r
Rank
Userlevel 5

To avoid floating point issues, try using the StringReplacer to simply remove any decimal point and everything that follows at the end of the value:

david_r
Rank
Userlevel 5

Someone told me about floating point issues a while back :-)

https://knowledge.safe.com/questions/3538/attribute-rounder-0145.html

I should definitely follow this guy... ;-)

jakethepainter
Rank
Badge +5

Thanks everyone. Looks like the string functions may be better for this.

mark2atsafe
Rank
Userlevel 4
Badge +25

I'll file this with the devs. I know it's expected from the computing point of view, but I'm sure we must be able to come up with a fix for this - at least in the AttributeRounder. Using a direct function might be a different matter.

mark2atsafe
Rank
Userlevel 4
Badge +25

So I made this my question-of-the-week.

While writing that it occurred to me that perhaps you could just use the round() function instead of int(), and then put the data through a DateTimeConverter with Repair Overflow turned on.

If you just want valid datetimes, rounded to the nearest second (up or down), that would be the way to go I think.

But if you absolutely want them rounded down/truncated, then the StringReplacer is the better solution.

Reply


Community Stats

28,783
Posts
109,736
Replies
36,608
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.