Skip to main content
Archived

Add Change Data Capture (CDC) Capabilities at the Database Log Level

Related products:FME Form
siennaatsafe
fmelizard
  • siennaatsafe
    siennaatsafe
  • fmelizard
    fmelizard

fmelizard
Safer

It would be great if FME could monitor log files of one database, and when a transaction occurs, relay that change to another type of database in real time.

This post is closed to further activity.
It may be a question with a best answer, an implemented idea, or just a post needing no comment.
If you have a follow-up or related question, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

3 replies

  • August 9, 2018

Can't seem to add any significant comment without it disappearing into the aether. joseph.healy@nwnatural.com


  • August 9, 2018

Database log-level change detection is quite a bit different than querying a database to detect changes which is what the below seems to be what the link you sent is about.

Another way to approach this might piggy-backing on top of / partnering with one of the CDC vendors I listed. They can all detect and replicate changes in Oracle, SQL Server, Postgres, etc., but I suspect they have no idea how to decode that data or how all the changes might be related – maybe you could do some sort of plugin for them:

https://www.hvr-software.com/wiki/AgentPlugin

All these companies do the same for SAP data which, while installed on common databases, have fairly indecipherable record structures and relationships and a lot of their value and sales is driven by their ability to decode SAP data and make it usable in other databases. You guys could do essentially the same thing for ESRI/Geospatial data because the CDC vendors either aren’t interested or don’t have the expertise to do that work and I’m in discussions with them. Maybe you should be as well…

Joseph Healy

NW Natural – Application Architect


  • August 9, 2018

CDC isn’t an ESRI thing, it’s an generic integration technology where CDC (Change Data Capture) software monitors the log files of one database and anytime a database transaction is registered to the source log the CDC software captures that change and relays it in real time to a different database - typically any of many different types of target databases,

See these vendors:

https://www.attunity.com/real-time-data-integration/

https://www.hvr-software.com/solutions/database-replication/

https://theobald-software.com/en/

We don’t want to run batch or scheduled jobs against ESRI, we want any change to the ESRI database to be relayed to another database (say SQL Server) in real time.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings