Skip to main content
Best Answer

Updating data using incremental Updates

  • July 9, 2020
  • 1 reply
  • 71 views
gisgeek
Contributor
Forum|alt.badge.img+9

I am trying to set up a process where I have a bulk dataset of approximately 77,000 records and will be needing to do weekly updates. The first incremental update had 48 records so I am trying to do UPDATE, INSERT, DELETE of these to my bulk dataset with 77,000 records. I have set up a process that looks like this. How do I compare the records before I write it out though and is this the best way to do it but can't seem to get this process to work using change detector. When I looked at the bulk record it had only added 12 records but thought I would get 13 extra records based on my workbench.

Best answer by markatsafe

@gisgeek Does your CSV file already have an update type (insert, update, delete)? if it does, then what it looks like you're doing in the image should work.

In the AttributeCreator set fme_db_operation to be INSERT|UPDATE|DELETE and on the writer feature type set Feature Operation: fme_db_operation and set the Match Columns attributes (keys). More details in the FME Documentation and this tutorial

You shouldn't need the ChangeDetector if your input data has the update flags (insert,update,delete)

If it's still not working, share a small sample dataset.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

1 reply

M
Forum|alt.badge.img+2
  • markatsafe
  • Best Answer
  • July 10, 2020

@gisgeek Does your CSV file already have an update type (insert, update, delete)? if it does, then what it looks like you're doing in the image should work.

In the AttributeCreator set fme_db_operation to be INSERT|UPDATE|DELETE and on the writer feature type set Feature Operation: fme_db_operation and set the Match Columns attributes (keys). More details in the FME Documentation and this tutorial

You shouldn't need the ChangeDetector if your input data has the update flags (insert,update,delete)

If it's still not working, share a small sample dataset.

Community Stats

32,182
Posts
122,179
Replies
39,856
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.