Skip to main content
Question

Find similar records and merge them to one


messagemauri
Contributor
Forum|alt.badge.img+6

I have a table called PROJECTS with some records sharing similar names. The objective is to find these records and merge them to one if possible.

 

Sample

PROJECTS table with Field1, Field2, Field3

 

FindSimilar 

From the above table, I want to look into Field3 and find similar names, such as Paper Company, Paper Company. and Paper Co and merge these into a single record. They are not exactly duplicates, but share a similar name. There are more records with similar names, so I can't hard-code Paper Company because there are more records alike that share similar names.

 

What transformers can I use to complete such task?

 

Thank you.

 

3 replies

danilo_fme
Evangelist
Forum|alt.badge.img+45
  • Evangelist
  • March 4, 2022

Hi @messagemauri​ 

 

I simulated here your case and had a great results using the custom transformer FuzzyDuplicateRemover:

Workspace_Duplicate 

As you can see, I configurated the ratio of similarity for 70 %.

Results - Unique:

UniqueResults - Duplicate:

Duplicate 

 

Thanks in Advance,

Danilo


messagemauri
Contributor
Forum|alt.badge.img+6
  • Author
  • Contributor
  • March 4, 2022
danilo_fme wrote:

Hi @messagemauri​ 

 

I simulated here your case and had a great results using the custom transformer FuzzyDuplicateRemover:

Workspace_Duplicate 

As you can see, I configurated the ratio of similarity for 70 %.

Results - Unique:

UniqueResults - Duplicate:

Duplicate 

 

Thanks in Advance,

Danilo

Thank you @danilo_fme​ . This will give me a starting point. 👍


danilo_fme
Evangelist
Forum|alt.badge.img+45
  • Evangelist
  • March 4, 2022
messagemauri wrote:

Thank you @danilo_fme​ . This will give me a starting point. 👍

Awesome! Thanks your feedback @messagemauri​ 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings