Skip to main content
Question

AttributeRounder Error/Warning


mel
Contributor
Forum|alt.badge.img+6
  • Contributor
When using Attribute rounder with ArcGIS 10.1 /Oracle 11 we get the following warning when we use the attribute rounder. We are generally rounding to either 0, 1, or 2 decimal places.

 

 

AttributeRounder: High-precision rounding procedure resulted in overflow; original value kept

 

 

Does anyone know how to prevent this warning and why we get it and is this something we should worry about or is it safe to ignore it?

 

3 replies

david_r
Evangelist
  • March 24, 2013
Hi,

 

 

this warning message is not related to either ArcGIS or Oracle, you will sometimes see this message from an AttributeRounder regardless of how you use it. The reason for this message has always been a mystery to me, as rounding a floating point number correctly should be fairly easy.

 

 

That said, I've never really had any problems ignoring this warning. However, if you think this is causing problems in your process, consider doing the rounding with a PythonCaller. Example:

 

-----

 

def roundAttribute(feature):

 

    numberOfDigits = 2

 

    attributeToRound = "myAttributeNameHere"

 

    value = float(feature.getAttribute(attributeToRound))

 

    if value:

 

        feature.setAttribute(attributeToRound, round(value, numberOfDigits))

 

-----

 

This will round the attribute named "myAttributeNameHere" to 2 digits. Modify as needed.

 

 

Hope this helps.

 

 

David

fmelizard
Contributor
Forum|alt.badge.img+17
  • Contributor
  • March 25, 2013
Hi,

 

 

Another way to round is to make use of the round function in the expressionevaluator

david_r
Evangelist
  • March 25, 2013
Good call, Itay! Forgot about the ExpressionEvaluator.

 

 

Remember, though, that the Round() function alone will return an integer, i.e. you will lose all the decimals.

 

 

To round to the nearest 2 decimals, use this expression:

 

 

Round(@Value(myAttribute)*100)/100.0

 

Use 1000 for 3 decimals, etc.

 

David

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings