Question

Change Detector keeps identifying features as 'updated' even though the same exact data is being compared

T

I'm currently using the 'change detector' transformer to compare incoming data with an existing database, and my challenge is that the comparison always seems to spit out the features as being 'updated', even though I am testing by comparing essentially the same exact datasets, so I expect it should be exiting out as 'unchanged' instead. The inputs are coming from a Portal feature service and a geodatabase

 

When I toggle the 'Check geometry' off, it seems to work correctly and detect the features were 'unchanged', so I'm guessing it's some slight changes to the geometry that may be causing features to be seen as 'updated'.

 

I'm already using the 'coordinate rounder' to alleviate some slight variations, but are there other approaches I can try to help figure this out? Thanks!

13 replies

redgeographics
Rank
Userlevel 5
Badge +25

Under Geometry Matching you can try the Lenient geometry matching option (which will disregard any differences in the order of vertices) and a vector tolerance (will disregard minor variations). If that fails I'd take a very close look at a feature from the Geodatabase and the corresponding one from the Portal and see where they differ.

nielsgerrits
Rank
Userlevel 6
Badge +33

No hints in the logging when writing the updates?

No differences is dimension? (2D vs 3D)

It is hard to say without a sample of the data. Things you can try:

  • Isolate one feature in both sets to analyze.
  • Check for measures is both sets.
  • Copy the coordinate of an identical vertex in each set and paste to a text editor to compare the differences. You can do this using the Data Inspector.
  • Copy the WKT string from both features to notepad and compare. You can extract those using a GeometryExtractor.
  • In case of lines / polygons / buffers, check for differences in orientation. You can ignore orientation differences using the parameter "Lenient Geometry Matching".
david_r
Rank
Userlevel 4

I would advise against using the CoordinateRounder in your scenario, in some cases it may create more problem than it resolves, notably for values around the tipping point of either rounding up or down.

The proper solution would be to use Lenient Geometry Matching and the Vector Tolerance setting available in the ChangeDetector (newer versions of FME).

T

I would advise against using the CoordinateRounder in your scenario, in some cases it may create more problem than it resolves, notably for values around the tipping point of either rounding up or down.

The proper solution would be to use Lenient Geometry Matching and the Vector Tolerance setting available in the ChangeDetector (newer versions of FME).

Appreciate the guidance -- for the Vector Tolerance, what exactly are the units for that value or is there a good way to determine the number to use?

david_r
Rank
Userlevel 4

Appreciate the guidance -- for the Vector Tolerance, what exactly are the units for that value or is there a good way to determine the number to use?

The Vector Tolerance is in ground units, i.e. the same units as your coordinate system.

M
Badge +2

@tarikc​ If you select Generate Details Changes List for Update Detection and ad a list name, you'll get a detailed list of all the changes. If there is a geometry change, you'll see something like:

_changes{0}.action modified

_changes{0}.attributeName geometryObject

then you look at that feature in detail.

ebygomm
Rank
Userlevel 1
Badge +21

My suspicion would be that the minor changes to geometries are introduced when writing into the file gdb, which is why you see the changes when comparing between the portal features and the gdb

david_r
Rank
Userlevel 4

My suspicion would be that the minor changes to geometries are introduced when writing into the file gdb, which is why you see the changes when comparing between the portal features and the gdb

I can confirm that I've seen lots of cases like this, mostly linked to Esri Geodatabases. The likely cause is coordinate rounding operations done by the Esri layer due to the snapping grid defined in the feature class tolerance and precision settings.

M
Badge +2

I can confirm that I've seen lots of cases like this, mostly linked to Esri Geodatabases. The likely cause is coordinate rounding operations done by the Esri layer due to the snapping grid defined in the feature class tolerance and precision settings.

We definitely see this. One approach is to preserve the original geometry - GeometryExtractor - round the coords a little for ChangeDetector and then recover the original geometry with GeometryReplacer.

brublanc

My suspicion would be that the minor changes to geometries are introduced when writing into the file gdb, which is why you see the changes when comparing between the portal features and the gdb

I'm having the op's exact same issue, and I'm leaning towards this being the issue. What makes this the most infuriating is that both SDE databases/feature classes have identical settings (projection, geometry storage type, vector tolerance, etc.), yet this still happens... It's infuriating. And I don't have the option of converting these to shapefiles because my organization heavily uses SQL for GIS storage.

brublanc

I can confirm that I've seen lots of cases like this, mostly linked to Esri Geodatabases. The likely cause is coordinate rounding operations done by the Esri layer due to the snapping grid defined in the feature class tolerance and precision settings.

I'll try that and see if I can make it work.

 

Also, would you happen to know if Esri is even considering some type of fix for this? I know it's probably rock bottom in their priority list, but it's really disruptive for those of us relying on FME tools that use change detection.

M
Badge +2

@brublanc​ Have you tried using a Vector Tolerance on the ChangeDetector (under Check Geometry parameters)? I wonder if that will help in these cases.

B

I am having the exact same issue with ChangeDetector. From my experience, it's not just Esri's file geodatabase; I tried GeoPackage, SQL Spatial; they all randomly have this issue. My guess is when writing data, the geometry data changed. However, using "vector tolerance" will solve the issue, just put a really small number in there.

Reply


Community Stats

28,724
Posts
109,519
Replies
36,564
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.