I've written a workspace in the 2015 version of FME, with five readers, which all have one column headed 'REFERENCE'. The transformer 'DuplicateRemover' is producing two outputs: 'Unique' and 'Duplicate', but the inspectors reveal that both outputs contain unique values. In 'Duplicate', there are both duplicate values and unique values. Why is this, and how can I resolve this to produce only duplicate values in the 'Duplicate' output?
Solved
How does the 2015 transformer DuplicateRemover work?
Best answer by ebygomm
With the matcher you can choose no geometry matching
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
As far as I know the basic behavior hasn't changed in the recent versions, so here's from the documentation:
This transformer outputs to the Duplicate port any feature whose key attribute value is the same as one previously encountered. Any feature whose key value has not yet appeared is output via the Unique port.
The first feature with a unique key value is output via the Unique port – and subsequent features that have the same key value are output via the Duplicate port.
+23- Contributor
It works as a filter and is in FME 2016 renamed to DuplicateFilter (A fetaure is either Unique = first appearence or Duplicate = all other appearances).
See this link for more info: DuplicateFilter
Thank you all for your help. It makes sense now as to why my duplicate output is producing apparently unique values. Am currently working on using the matcher transformer. May need to get in touch again!
Unfortunately I'm still struggling with this issue. In essence, I want to produce a single list which features every instance of each duplicate, regardless of geometry.
Following the above discussion I rewrote my workspace, using the Matcher transformer, but found that it did not account for cases where my 'REFERENCE' attribute values were duplicated, but the geometry was unique.
Is there a non-geometry based version of the Matcher, or a way to use the Matcher but discount geometry?
If the answer to the above question is no, I feel I need a second transformer which connects the 'Matched' output to the 'NotMatched' output, and then compares the two to produce two outputs: one which contains all duplicate instances, and one which contains all cases which have no duplicates.
Thanks in advance.