I have a need to check for data changes between two datasets (oracle tables). I initially configured the ChangeDetector comparing all of the attributes. The results were, all of the data from the “revised” port were identified as records to be added (some 300K plus records) and the data from the “original” port are identified as records to be deleted (som 300K plus records).
There’s no unique key in this data. If I limit the number of attributes to check to 7 specific attributes, the “added” and “deleted” ports reflect more reasonable results. In other words, from a basic math perspective, the difference between the number of revised and original records calculate correctly (in the sense of finding “changes”).
Essentially, all of the attribute values can change so I need to check for all of the attributes. I don’t understand why in configuring the ChangeDetector to check for all attributes, why the transformer behaves as if all of the records “changed”. I’m fairly new to using this particular transformer.
I use to simply process all of the data which includes geocoding some 300 thousand plus records. I’m trying to build a more efficient process and relieve an anxiety from my GIS folks by using their geocoding web service to process 300 thousand plus records on a nightly basis.
Any thoughts?
