Question

Using the AttributeRounder does not always round to 3 decimal points.

D
Badge

When writing data to a zip shp. file often the data doesnt comply with the 3 decimal points put down in the CoordinateRounder. It usually ends in a couple decimals and then ending on 00001.

 

In the AttributeRounder description it states that this is an issue that can occur: The CoordinateRounder may not always behave as expected, due to known limitations in floating-point computing.

 

I am looking for a way to always make coordinates end on 3 decimals.

 

The data comes from an Oracle Spatial reader, which after some transformers, goes through a Generalizer and then a CoordinateRounder before writing the data as a zip shp.

 

Hopefully anyone here can help me out?

6 replies

M
Badge +2

@davey​ rounding floating point numbers in computing is a complicated issue. What is happening here is floating point limitation that is a common occurrence in computing. This is not restricted to FME and I have included PythonCaller in the example workspace and you can see that it also produces apparently inconsistent results. 

To us, the value 9.075 should obviously round to 9.08. But if you manipulate the value then you'll see the floating point representation. For example : if value = 9.075

value - int(value) = 0.07499999999999929 which would round down to 0.07

One approach is to simply check the 3rd digit and if this is >=5 your use ceil() and if <=4 you use floor(). I've illustrated this with the StringSearcher/AttriuteCreator in the attached workspace. 

It's very frustrating to what seems to us to be an obviously simple calculation.

 

For geometry, FME ALWAYS uses a floating point representation. So even if you round the geometry, when FME actually writes that to the target format, the value will revert to it's true floating point representation. So continuing with the example above, a coordinate that was rounded to 9.07, will revert to 9.07499999999999929.

Some formats, like Bentley MicroStation and Enterprise Geodatabase use integer coordinates, so in these cases the coordinates will 'snap' to the integer grid.

M
Badge +2

here's the promised example workspace (FME 2021)

D
Badge

@Mark Stoakes​ Thanks for your explanaition! If I understand the workspace and your description correctly there is not a way in FME to make a Geometry have a maximum of 3 decimals when it is Float point representation?

 

Even when we would use the StringSearcher/AttriuteCreator we would only be able to change the value of the attributes we created but not the geometry itself?

 

Would there be a so called 'work-around' to use the source data from a Enterprise Geodatabase instead of an Oracle Spatial database? This way you could manipulate the coordinates to behave as integer instead of floating? And if so, would it be possible to use the CoordinateRounder in this case?

 

The struggle for us is, we need to be able to upload a Geometry format to a website which can only handle a maximum of 3 decimal points in it's Coordinates.

tomfriedl
Rank
Userlevel 2
Badge +12

StringFormatter, Format String = .3f

https://community.safe.com/general-10/fme-form-starter-20721
ebygomm
Rank
Userlevel 1
Badge +10

@Mark Stoakes​ Thanks for your explanaition! If I understand the workspace and your description correctly there is not a way in FME to make a Geometry have a maximum of 3 decimals when it is Float point representation?

 

Even when we would use the StringSearcher/AttriuteCreator we would only be able to change the value of the attributes we created but not the geometry itself?

 

Would there be a so called 'work-around' to use the source data from a Enterprise Geodatabase instead of an Oracle Spatial database? This way you could manipulate the coordinates to behave as integer instead of floating? And if so, would it be possible to use the CoordinateRounder in this case?

 

The struggle for us is, we need to be able to upload a Geometry format to a website which can only handle a maximum of 3 decimal points in it's Coordinates.

I believe shape files store coordinates as floating points so there is no way round this if you have to use a shape file to upload

M
Badge +2

@Mark Stoakes​ Thanks for your explanaition! If I understand the workspace and your description correctly there is not a way in FME to make a Geometry have a maximum of 3 decimals when it is Float point representation?

 

Even when we would use the StringSearcher/AttriuteCreator we would only be able to change the value of the attributes we created but not the geometry itself?

 

Would there be a so called 'work-around' to use the source data from a Enterprise Geodatabase instead of an Oracle Spatial database? This way you could manipulate the coordinates to behave as integer instead of floating? And if so, would it be possible to use the CoordinateRounder in this case?

 

The struggle for us is, we need to be able to upload a Geometry format to a website which can only handle a maximum of 3 decimal points in it's Coordinates.

@davey​ I think the only way around this would be to use a text based format like GML, GeoJSON or even CSV. These essentially treat the coordinates as a string, so there is no floating point issue. For example, City of Vancouver Open Data Portal delivers CSV data with a a GeoJSON field for the coordinates.

Reply


Community Stats

28,661
Posts
109,318
Replies
36,528
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.