Skip to main content
Hi,

 

 

I've got a weird difference between x-y coordinates between two points which is causing some issues.

 

Using a FGDB, with a featureclass with points included, as a basemap I draw a polygon using the esri snapping function on. This polygon has its corners snapped to the points and is saved in a .shp shape file. In ArcMap the x-y coordinates of the corner points are exactly the same as the points underneath them.

 

 

Now when using FME and FME dataviewer these points have slightly different x-y coordinates only differing on the 9th decimal. Meaning that the points do not fall inside the polygon creating difficulties.

 

 

To keep it simple the x-coordinate of the point is:

 

67479.59299999848

 

and the x-coordinate of the polygon corner point is:

 

67479.593000002205

 

 

Now my question is this:

 

What makes the differences because the difference isn't just plain and simple rounding. With rounding I would expect a .593000000000 not a 2205 at the end.

 

 

Hope someone has an answer to how this works!

 

 

Cheers,

 

 

Hi

 

 

there are a few other threads in here that touch on the same issue, but the short answer is that this is due to internal rounding "errors" when converting fractions between base 10 (you and me) and base 2 (the computer). It should not make any difference for most applications.

 

 

You can find a more detailed answer here: http://floating-point-gui.de/basic/

 

 

David
Thank you that was the answer I was looking for. So far I hadn't found any answers that answered to satisfaction but I haven't read every thread here 😃.

 

Sorry for any duplication!

Reply