Skip to main content
Question

@int function is rounding up in some cases

  • February 27, 2020
  • 9 replies
  • 131 views

jakethepainter
Contributor
Forum|alt.badge.img+6

FME 2019.2 desktop user. I have some dates that have fractional seconds and I want to truncate the fractional seconds to whole seconds. In the fme help for the arithmetic editor it says the @int function converts by truncation if it is in the range of a 64 bit integer, which is exactly what I want.

Most dates I'm sending in are truncated. But ones of the type 20200220015759.999000 are getting rounded up. Which, in the case of above, is an invalid date because it rounds up to 60 seconds.

Is this a bug or am I missing something?

 

9 replies

david_r
Celebrity
  • February 27, 2020

I agree that this is unexpected behavior, I was able to reproduce it as well. Even the AttributeRounder set to round down with 0 decimals gave the wrong result.

Unless someone comes up with a good workaround, consider forwarding the case to Safe support, pointing back to this thread.


ebygomm
Influencer
Forum|alt.badge.img+32
  • Influencer
  • February 27, 2020
david_r wrote:

I agree that this is unexpected behavior, I was able to reproduce it as well. Even the AttributeRounder set to round down with 0 decimals gave the wrong result.

Unless someone comes up with a good workaround, consider forwarding the case to Safe support, pointing back to this thread.

Isn't this a floating point issue? The following in python returns 20200220015760.0


david_r
Celebrity
  • February 27, 2020
ebygomm wrote:

Isn't this a floating point issue? The following in python returns 20200220015760.0

Looks like you're right:

and


ebygomm
Influencer
Forum|alt.badge.img+32
  • Influencer
  • February 27, 2020
david_r wrote:

Looks like you're right:

and

Someone told me about floating point issues a while back :-)

https://knowledge.safe.com/questions/3538/attribute-rounder-0145.html


david_r
Celebrity
  • February 27, 2020

To avoid floating point issues, try using the StringReplacer to simply remove any decimal point and everything that follows at the end of the value:


david_r
Celebrity
  • February 27, 2020
ebygomm wrote:

Someone told me about floating point issues a while back :-)

https://knowledge.safe.com/questions/3538/attribute-rounder-0145.html

I should definitely follow this guy... ;-)


jakethepainter
Contributor
Forum|alt.badge.img+6
  • Author
  • Contributor
  • February 27, 2020

Thanks everyone. Looks like the string functions may be better for this.


mark2atsafe
Safer
Forum|alt.badge.img+44
  • Safer
  • February 28, 2020

I'll file this with the devs. I know it's expected from the computing point of view, but I'm sure we must be able to come up with a fix for this - at least in the AttributeRounder. Using a direct function might be a different matter.


mark2atsafe
Safer
Forum|alt.badge.img+44
  • Safer
  • February 28, 2020

So I made this my question-of-the-week.

While writing that it occurred to me that perhaps you could just use the round() function instead of int(), and then put the data through a DateTimeConverter with Repair Overflow turned on.

If you just want valid datetimes, rounded to the nearest second (up or down), that would be the way to go I think.

But if you absolutely want them rounded down/truncated, then the StringReplacer is the better solution.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings