Dividing Integers, Managing Number Types
How do I specify what type of number I want for arithmetic operations? For example, when use the Attribute Creator and then the Arithmetic Editor to divide two numbers that are integers, the result is a rounded/truncated integer, but I want a number that is 3 decimal places...
Userlevel 4
+13
- Contributor
- 3725 replies
-
15 November 2012
Hi,
The ExpressionEvaluator documentation states:
"5 / 4
returns 1, while
5 / 4.0
and
5 / (4 + 0.0)
both return 1.25.
Hope this helps.
+22
- Contributor
- 1961 replies
-
15 November 2012
if you're using any constants in the expresion make them explicitly real numbers (ex. 2.0 instead of 2), if not multiply the entire expression by 1.0.
After your expression evalutaor, use an AttributeRounder with the number of decimal places set to 3
Userlevel 4
+13
- Contributor
- 3725 replies
-
16 November 2012
... plus you can explictly call @double( ) to ensure that whatever argument in there is treated as a double .. . (in the math editor). And be sure, as Itay suggests, to add a .0 to any constants you are working with...
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.