I have two datasets with 30 columns and after running the changedetector to compare 20 of these columns it states 0 are unchanged, how can I figure out which of these columns is the cause for the datasets not being the same?
Would I use featuremerger or is there a simpler solution? to compare the columns to see whats different based on 1 shared unchanged column.
Hi @richard_che,
In the ChangeDetector there is the option to Generate a Detailed Change List:
By specifying a List Name this will then generate a list that includes the action, the attribute name, the original and revised value for all changes associated with this feature, for example in the image below attr1 value changed from 0 to 2 and attr2 value was removed entirely:
Hopefully this list will give you the comparison you are looking for, you can use the ListExpoder transformer after the ChangeDetector if you want to bring these list elements out into Attributes.
Hi @richard_che,
In the ChangeDetector there is the option to Generate a Detailed Change List:
By specifying a List Name this will then generate a list that includes the action, the attribute name, the original and revised value for all changes associated with this feature, for example in the image below attr1 value changed from 0 to 2 and attr2 value was removed entirely:
Hopefully this list will give you the comparison you are looking for, you can use the ListExpoder transformer after the ChangeDetector if you want to bring these list elements out into Attributes.
@hollyatsafe, this option is not yet showing in the documentation in the transformer gallery. Since it is quite a great addition, updating the documentation would be of high value. Thanks.
Hi @richard_che,
In the ChangeDetector there is the option to Generate a Detailed Change List:
By specifying a List Name this will then generate a list that includes the action, the attribute name, the original and revised value for all changes associated with this feature, for example in the image below attr1 value changed from 0 to 2 and attr2 value was removed entirely:
Hopefully this list will give you the comparison you are looking for, you can use the ListExpoder transformer after the ChangeDetector if you want to bring these list elements out into Attributes.
Thank you Holly! That definitely helped. I'll admit I'm a bit of a novice with FME and did generate a list, but do not see it visible in the visual preview or when choosing the columns to preview. I did attempt to use the listexploder however couldnt not find the generated list name when trying to run that transformer.
My output looks as seen below
@hollyatsafe, this option is not yet showing in the documentation in the transformer gallery. Since it is quite a great addition, updating the documentation would be of high value. Thanks.
Hi @erik_jan,
I see this already listed in our most current Documentation http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/changedetector.htm, however the Transformer Gallery will link to the Documentation for your version of FME Workbench. It looks like this was parameter was new to this transformer for 2019.0 but the documentation update only went into 2019.2, sorry about that!
Hi @erik_jan,
I see this already listed in our most current Documentation http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/changedetector.htm, however the Transformer Gallery will link to the Documentation for your version of FME Workbench. It looks like this was parameter was new to this transformer for 2019.0 but the documentation update only went into 2019.2, sorry about that!
If I go to the online transformer gallery: Transformer Gallery
And the to the documentation for the ChangeDetector: ChangeDetector
I do not see the list mentioned.
Thank you Holly! That definitely helped. I'll admit I'm a bit of a novice with FME and did generate a list, but do not see it visible in the visual preview or when choosing the columns to preview. I did attempt to use the listexploder however couldnt not find the generated list name when trying to run that transformer.
My output looks as seen below
You should see the list in the feature information window, it does not appear in the table view.
You should see the list in the feature information window, it does not appear in the table view.
I had opened the data inspector did not see the column "PA20" listed anywhere there, and also checked the feature information window (its not there either).
I'm not an expert with FME but I've worked with lists before, and usually you can see them in the visual preview when inspecting the data if you hit columns (it will show your dataset attributes, and will also show lists, I did not see any lists for the final output). I believe thats because the data is essentially not being matched at all, as there is nothing going to Updated, it all goes to inserted and deleted.
I guess I could try feature merger and compare it column by column since they both share a unique identifier.
EDIT:
I also cleaned up the dataset with only the information I really need to compare and theres still no list being generated.
Hi @richard_che,
In the ChangeDetector there is the option to Generate a Detailed Change List:
By specifying a List Name this will then generate a list that includes the action, the attribute name, the original and revised value for all changes associated with this feature, for example in the image below attr1 value changed from 0 to 2 and attr2 value was removed entirely:
Hopefully this list will give you the comparison you are looking for, you can use the ListExpoder transformer after the ChangeDetector if you want to bring these list elements out into Attributes.
And if you add the attributes you want to compare in the update detection key attributes parameter? Still no results?
If I go to the online transformer gallery: Transformer Gallery
And the to the documentation for the ChangeDetector: ChangeDetector
I do not see the list mentioned.
Ah, sorry I thought you meant the Transformer Gallery Help through workbench. Woops yes it looks like our website is still pointing to the 2019 documentation, I'll get that fixed.
To see the most up-to-date documentation simply remove 2019.0/ from the URL.
I had opened the data inspector did not see the column "PA20" listed anywhere there, and also checked the feature information window (its not there either).
I'm not an expert with FME but I've worked with lists before, and usually you can see them in the visual preview when inspecting the data if you hit columns (it will show your dataset attributes, and will also show lists, I did not see any lists for the final output). I believe thats because the data is essentially not being matched at all, as there is nothing going to Updated, it all goes to inserted and deleted.
I guess I could try feature merger and compare it column by column since they both share a unique identifier.
EDIT:
I also cleaned up the dataset with only the information I really need to compare and theres still no list being generated.
Ha I see it now....the list is only available for updated features and not inserted (compleatly new features in the revised dataset)
And if you add the attributes you want to compare in the update detection key attributes parameter? Still no results?
Yes that was the problem! Thank you, I had left it empty (as you can see in the picture above), filling that field generated the list and I can view the differences. (Thankfully they're geometry issues).
Thank you Holly! That definitely helped. I'll admit I'm a bit of a novice with FME and did generate a list, but do not see it visible in the visual preview or when choosing the columns to preview. I did attempt to use the listexploder however couldnt not find the generated list name when trying to run that transformer.
My output looks as seen below
Hi @richard_che,
Ah I see all of your features are coming out of the inserted/deleted ports. The Generate a Detailed Change List is only applicable to the Updated port which is why you do not see a list feature at all.
I'm wondering if you need to actually set the 'Update Detection Key Attributes' parameter.
This allows the features from the Original and Revised ports to be joined before they are compared. If a joined pair is identical, the feature from the Revised port will go out the Unchanged port. If a joined pair differs, the feature from the Revised port will go out the Updated port.
If you leave this unset you'll see features either inserted or deleted but no additional information as it's reporting the entire feature as inserted or deleted, not a change to a specific attribute on that feature.
If I go to the online transformer gallery: Transformer Gallery
And the to the documentation for the ChangeDetector: ChangeDetector
I do not see the list mentioned.
Hi @erik_jan
Just a quick update to let you know the online Transformer Gallery now points to the most recent version of our documentation, thanks for pointing that out to us!