Skip to main content
Solved

How to limit the decimals in the result of the lengthcalculator ?


Hello,

I have a shp layer of lines for which I need to calculate the length of each line then round it to suppress the decimals.

 

I use the lenghtcalculator to get the lenght of each line but for some lines I get results such as 7.566995918750763e-10 or 1.7276990785794134e-9

With these results, I get 0 after the attributerounder. I even tried a stringformatter to reduce the amount of decimals but theses objets also turn into 0.

Is there a way to avoid getting so much precision when calculating the lenght ? Or another way to calculate the lenght (I get the same result with an attribute creator and length() ) ?

 

Many thanks in adance!

Ann

Best answer by david_r

7.566995918750763e-10 (scientific notation) is the same as 0.0000000007566995918750763, so unless you round down to more than 9 decimals the result is actually going to be 0.

I'm guessing these are extremely short line segments?

View original

david_r
Evangelist
  • October 9, 2020

7.566995918750763e-10 (scientific notation) is the same as 0.0000000007566995918750763, so unless you round down to more than 9 decimals the result is actually going to be 0.

I'm guessing these are extremely short line segments?


ebygomm
Influencer
Forum|alt.badge.img+29
  • Influencer
  • October 9, 2020

What distance units are you working in, 7.566995918750763e-10 is a very small number which is why it rounds to zero


Thank you very much. I understand better what it means, it is logical that it rounds to 0. Indeed these are extremely short lines. Thanks!


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings