Skip to main content
Solved

How can I remove duplicates (filtered with other attributes)

wolters
Contributor
Forum|alt.badge.img+6

Hello everyone, and thank you for looking!

 

I am asking your help on an operation that I would like to do, but I am unsure how to do it in FME.

 

I have a table such as this:

 

ID AREA VILLAGE

1 232 Oxenfurt

1 452 Las Vegas

1 923 New York

2 124 Jacksonville

2 298 Charlotte

3 767 Las Vegas

3 422 Charlotte

3 223 Oxenfurt

etc...

 

What I want to get is table where ID is unique with a single VILLAGE entry, selected based on the maximum value of the AREA entry, and all other entries with the same ID removed. Taking the above table as example, the result that I would like would be:

 

ID AREA VILLAGE

1 923 New York

2 298 Charlotte

3 767 Las Vegas

etc...

 

How do I achieve this? Note that the input also has polygons, but those do not matter; I want to only have a resulting list of unique ID with the VILLAGE that has the highest AREA only, and delete the others.

 

Best answer by bwn

1. Sort the Features with a Sorter by Area: Numeric Descending

2. Send the Output of this to DuplicateFilter, with ID as the Attribute to test

 

DuplicateFilter will always hold onto the first Feature with the unique Attribute value combination as the "Unique" Feature and all others as "Duplicates". If you use a Sorter to make sure that the Top result you want is the first feature into DuplicateFilter out of a Group of duplicates, then this is what is guaranteed to come out the Unique Port.

 

View original
Did this help you find an answer to your question?

3 replies

bwn
Evangelist
Forum|alt.badge.img+26
  • bwnEvangelist
  • Evangelist
  • Best Answer
  • October 16, 2020

1. Sort the Features with a Sorter by Area: Numeric Descending

2. Send the Output of this to DuplicateFilter, with ID as the Attribute to test

 

DuplicateFilter will always hold onto the first Feature with the unique Attribute value combination as the "Unique" Feature and all others as "Duplicates". If you use a Sorter to make sure that the Top result you want is the first feature into DuplicateFilter out of a Group of duplicates, then this is what is guaranteed to come out the Unique Port.

 

ottadini
1 person upvoted this
  • ottadini
    ottadini
wolters
Contributor
Forum|alt.badge.img+6
  • woltersContributor
  • Author
  • Contributor
  • October 18, 2020

This is exactly what I wanted. I knew about the existence of DuplicateFilter, but was not aware that it always takes the first entry and deletes the rest. Thank you very much!

bwn
Evangelist
Forum|alt.badge.img+26
  • bwnEvangelist
  • Evangelist
  • October 18, 2020
wolters wrote:

This is exactly what I wanted. I knew about the existence of DuplicateFilter, but was not aware that it always takes the first entry and deletes the rest. Thank you very much!

A more general tip with FME is that for many Transformers their behavior and output is dependent on the order that the Features enter the Transformer, so Sorter is a very common Feature Order controller used in front of other Transformers to make sure particular features enter first or last. For example, LineBuilder for controlling the direction of lines generated, AreaOnAreaOverLayer is you set Attribute Accumulation to be "Use Attributes From One Feature" etc. to control which Feature's Attribute values take precedence etc.

Reply


Helpful Members This Week

Community Stats

31,378
Posts
119,013
Replies
39,009
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.