Skip to main content
Solved

AttributeManager Arithmetic Editor - Testing floats for equality?

  • September 10, 2021
  • 2 replies
  • 17 views

miczat
Contributor
Forum|alt.badge.img+4

How in FME would you check that two floats are "close", given testing floats for equality can be a challenge?

 

e.g. "-23.123" as listed in an XLS compared to "-23.12300005" stored in a point feature class?

 

In Python I use the math.isClose() or assertAlmostEqual()

 

Thanks

 

Best answer by debbiatsafe

math.isClose() uses this formula to evaluate whether two values are close or not:

abs(a-b) <= max( rel_tol * max(abs(a), abs(b)), abs_tol )

You can try implementing this formula within an AttributeManager using math operators and math functions. Using the default values for rel_tol and abs_tol, it would look something like this:

@Evaluate(@abs(@Value(fc)-@Value(xls)) <= @max(@exp(-09) * @max(@abs(@Value(fc)), @abs(@Value(xls))), 0.0))

A result of 1 would be true (close). 0 would indicate false (not close). I hope it helps!

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

debbiatsafe
Safer
Forum|alt.badge.img+21
  • Safer
  • 648 replies
  • Best Answer
  • September 10, 2021

math.isClose() uses this formula to evaluate whether two values are close or not:

abs(a-b) <= max( rel_tol * max(abs(a), abs(b)), abs_tol )

You can try implementing this formula within an AttributeManager using math operators and math functions. Using the default values for rel_tol and abs_tol, it would look something like this:

@Evaluate(@abs(@Value(fc)-@Value(xls)) <= @max(@exp(-09) * @max(@abs(@Value(fc)), @abs(@Value(xls))), 0.0))

A result of 1 would be true (close). 0 would indicate false (not close). I hope it helps!


miczat
Contributor
Forum|alt.badge.img+4
  • Author
  • Contributor
  • 5 replies
  • September 13, 2021

Thanks Debbi!