Skip to main content
Question

Attribute Creator converts String to Incorrect (Floating-Point) Number

  • March 19, 2019
  • 2 replies
  • 26 views
P
Forum|alt.badge.img

When I use an Attribute Creator to create a "copy" of an Attribute, it converts the value to an incorrect Floating-Point number. 92.7 --> 92.700000000000003. When I compare this value later, the test obviously fails. How do I ensure the correct number (or the string) is retained?

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

david_r
Celebrity
  • david_rCelebrity
  • March 19, 2019

This is not necessarily an error, see https://floating-point-gui.de/

If it's an issue for your use case, consider using an AttributeRounder.

P
Forum|alt.badge.img
  • peterx
  • Author
  • March 20, 2019

This is not necessarily an error, see https://floating-point-gui.de/

If it's an issue for your use case, consider using an AttributeRounder.

How do I force the use of a decimal data-type or prevent the numeric conversion?

You'll note the issue is also quite misleading in the inspector as per the screen-shot above.

Community Stats

32,120
Posts
121,954
Replies
39,772
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.