Question
Dividing Integers, Managing Number Types
How do I specify what type of number I want for arithmetic operations? For example, when use the Attribute Creator and then the Arithmetic Editor to divide two numbers that are integers, the result is a rounded/truncated integer, but I want a number that is 3 decimal places...
+18- Safer
Hi,
The ExpressionEvaluator documentation states:
"5 / 4
returns 1, while
5 / 4.0
and
5 / (4 + 0.0)
both return 1.25.
Hope this helps.
+28- Contributor
if you're using any constants in the expresion make them explicitly real numbers (ex. 2.0 instead of 2), if not multiply the entire expression by 1.0.
After your expression evalutaor, use an AttributeRounder with the number of decimal places set to 3
+18- Safer
... plus you can explictly call @double( ) to ensure that whatever argument in there is treated as a double .. . (in the math editor). And be sure, as Itay suggests, to add a .0 to any constants you are working with...
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.