How to remove the repeated words in FME

Hi @chaoluo 

You can use this logic:

@chaoluo  Similarly method would use identical to that proposed by @nielsgerrits , @danilo_fme  and @geomancer 

The only difference between the methods is whether to use ListExploder + DuplicateFilter after the AttributeSpliiter, or instead just use ListDuplicateRemover after the AttributeSplitter.

I’ve used both approaches, and it depends on how big the Lists become over how many Features.  ListExploder + DuplicateFilter , despite needing more Transformers, can execute faster, as ListDuplicateRemover can be slow to traverse all the List Attributes and it has an overhead in having to rename whatever List Attributes are left after the duplicate all to new List index numbers.

So small->medium number of features and not a lot duplicates, ListDuplicateRemover approach above works fine as general approach, but if it performs slowly can look to trial the ListExploder + DuplicateFilter method instead.

The only extra tip is to think about an extra Sorter before the Aggregator (@nielsgerrits method) or ListSorter before the ListConcantenator ( @danilo_fme , @geomancer ) to get the values alphabetically sorted, comma-delimited in the final output.  I do this a lot to reduce the amount of random ordering that flows into say a ChangeDetector where if I didn’t sort the list first before comma-delimiting it, ChangeDetector would keep flagging a record had “changed”, where the next run of the workspace slightly change order of values, but where otherwise the same text strings and would cause the write to database to have an excess number of updates needed only because of the sometimes randomness of the order in fields with Eg. comma-delimited values.