• EN
Solved

Dynamic Update Detector

  • 19 December 2018
  • 3 replies
  • 2 views
oliver.morris
Rank
Badge +10

Hi, I was wondering if anyone has created a dynamic update detector. Ideally where you pass the original table, revised table, key attribute and attributes to match as parameters to a workspace or custom transformer.

something along the lines of:

At the moment I would need to have all the tables dynamically popualted along with the key attribute and attributes to match. This is because when I tried to do it with defined tables I run out of memory, so it would be more ideal to do it one at a time. This could then be super reusable in all code.

 

Many Thanks for any help or pointers.

 

Oliver

icon

Best answer by markatsafe 20 December 2018, 17:35

View original

3 replies

M
Badge +2

We've re-engineered UpdateDetector in FME 2019. It has been merged into the ChangeDetector and it is no longer a custom transformer but a fully fledged member of the transformer community. You can download FME 2019 betas from here if you want to give it a try. You can wrap the new ChangeDetector/UpdateDetector in a custom transformer to add a group by so each set of features could be grouped as described in this article.

(at the time of writing there are still some interface layout changes to be made on the new ChangeDetector/UpdateDetector to make it easier to use)

oliver.morris
Rank
Badge +10

We've re-engineered UpdateDetector in FME 2019. It has been merged into the ChangeDetector and it is no longer a custom transformer but a fully fledged member of the transformer community. You can download FME 2019 betas from here if you want to give it a try. You can wrap the new ChangeDetector/UpdateDetector in a custom transformer to add a group by so each set of features could be grouped as described in this article.

(at the time of writing there are still some interface layout changes to be made on the new ChangeDetector/UpdateDetector to make it easier to use)

Thanks Mark, I have tried to make it dynamic but alas I am not getting much further, please check out the template file, I can pass the list and the matching field but it doesnt seem to match them correctly. My only other thought it to parameterise them all then just use a workspace runner?

Thanks for any guidance you may have. The issue is I cant run this for all tables because the memory simply fills up and then fails, I need to do it one at a time. At the moment I simply truncate and populate the table but this is not that smart especially when the sql jdbc reader/writer for sql server is so slow.

Many Thanks reworked auto detector.fmwt

Oliver

oliver.morris
Rank
Badge +10

We've re-engineered UpdateDetector in FME 2019. It has been merged into the ChangeDetector and it is no longer a custom transformer but a fully fledged member of the transformer community. You can download FME 2019 betas from here if you want to give it a try. You can wrap the new ChangeDetector/UpdateDetector in a custom transformer to add a group by so each set of features could be grouped as described in this article.

(at the time of writing there are still some interface layout changes to be made on the new ChangeDetector/UpdateDetector to make it easier to use)

thanks @MarkAtSafe if we can have something more along the lines of merge - https://docs.microsoft.com/en-us/sql/t-sql/statements/merge-transact-sql?view=sql-server-2017 that would be amazing, look forward to checking out the amendments in 2019.

Reply


Community Stats

28,701
Posts
109,460
Replies
36,550
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.