Skip to main content
  • EN

Hello,

 

We have been troubleshooting this FME translation for a while and can’t figure out a solution.  The Inserted, Deleted features of the ChangeDetector write reasonably quickly but the Update features are writing so slow it will be unusable, GIF for reference. 

 

Things we’ve tried without any differences:

Indexing all fields in ArcCatalog

Indexing Just the Match Column fields (3 Fields to make a Unique ID)

Indexing and using a CRCCalculator field for the Match Columns

Adding a FeatureHolder after the ChangeDetector

Adding a AttributeManager after the ChangeDetector removing unused fields

Writing to a File Geodatabase (SDE is ultimately the goal and being used in the gif below)

Not Checking the Geometry in the ChangeDetector

Changing Transaction types in the Writer

Changing and testing various other settings in the transformers and writers

 

 

Thank you for any suggestions!!

To be honest, this is hard to debug via a forum. But my gut feeling is it has to do with the 3 fields to match. I would try and test if using one field, OBJECTID, only and see if performance is OK. When this is the case, merge OBJECTID to the feature in workbench. This is how I do it in my workspaces on SDE.

What FME version do you use?

I’m pretty confident it’s SDE slowing down the process. Updates are generally pretty slow in SDE land, I’m also guessing you’re probably using a versioned feature class? - to be clear if this is the case, the issue lies with SDE/Esri, not FME

 

One way to confirm its the updating is to add a feature holder between the change detector and the writer. If all features pass through the change detector then slowly filter out of the feature holder then it confirms SDE is slowing it down



 

Is there anyway to use a single field as the unique identifier?  Also, be aware of the following for the CRCCalculator:

“Like all popular CRC algorithms, different features could produce the same CRC value with minimal probability.”

To be honest, this is hard to debug via a forum. But my gut feeling is it has to do with the 3 fields to match. I would try and test if using one field, OBJECTID, only and see if performance is OK. When this is the case, merge OBJECTID to the feature in workbench. This is how I do it in my workspaces on SDE.

What FME version do you use?

 

So using OID isn't an option for our use case but using it as a test did help us get on the right path.  The problem was that our match columns weren't producing Unique rows as we had thought.  We were able to add another column to the Match Columns (Now 4 total fields that we are checking instead of the 3 I mentioned in my post) that would provide unique rows and everything ran correctly.  Thank you for your help!

Reply


Community Stats

30,362
Posts
115,380
Replies
38,092
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.