Skip to main content
Solved

AttributeManager Arithmetic Editor - Testing floats for equality?

  • September 10, 2021
  • 2 replies
  • 6 views

miczat
Contributor
Forum|alt.badge.img+3

How in FME would you check that two floats are "close", given testing floats for equality can be a challenge?

 

e.g. "-23.123" as listed in an XLS compared to "-23.12300005" stored in a point feature class?

 

In Python I use the math.isClose() or assertAlmostEqual()

 

Thanks

 

Best answer by debbiatsafe

math.isClose() uses this formula to evaluate whether two values are close or not:

abs(a-b) <= max( rel_tol * max(abs(a), abs(b)), abs_tol )

You can try implementing this formula within an AttributeManager using math operators and math functions. Using the default values for rel_tol and abs_tol, it would look something like this:

@Evaluate(@abs(@Value(fc)-@Value(xls)) <= @max(@exp(-09) * @max(@abs(@Value(fc)), @abs(@Value(xls))), 0.0))

A result of 1 would be true (close). 0 would indicate false (not close). I hope it helps!

View original
Did this help you find an answer to your question?

2 replies

debbiatsafe
Safer
Forum|alt.badge.img+20
  • Safer
  • Best Answer
  • September 10, 2021

math.isClose() uses this formula to evaluate whether two values are close or not:

abs(a-b) <= max( rel_tol * max(abs(a), abs(b)), abs_tol )

You can try implementing this formula within an AttributeManager using math operators and math functions. Using the default values for rel_tol and abs_tol, it would look something like this:

@Evaluate(@abs(@Value(fc)-@Value(xls)) <= @max(@exp(-09) * @max(@abs(@Value(fc)), @abs(@Value(xls))), 0.0))

A result of 1 would be true (close). 0 would indicate false (not close). I hope it helps!


miczat
Contributor
Forum|alt.badge.img+3
  • Author
  • Contributor
  • September 13, 2021

Thanks Debbi!


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings