Question

Replacing duplicate list elements with a blank space but leave blanks and and maintain element count..?

  • 12 March 2019
  • 7 replies
  • 6 views

Badge +1

I have a nine element, pipe-delimited attribute. These elements can be a text string or a blank space. I want to replace duplicate text elements with a blank space and leave blank elements alone. I.e. have it come out with eight delimiters and nine elements when I put it back together.

ListDuplicateRemover gets rid of the unwanted text elements, but also the blank ones.

 

Example:

|||Carnegie Drive|Carnegie Drive||Dunfermline||Ky12 7bd

when run through AttributeSplitter and ListDuplicateRemover becomes:

|Carnegie Drive|Dunfermline|Ky12 7bd

But I want it to be ' ||||Carnegie Drive||Dunfermline||Ky12 7bd' (with a leading space)

I.e., just remove the duplicate instance of 'Carnegie Drive' and replace it with a blank space.

Thoughts? Thanks


7 replies

Userlevel 1
Badge +21

Explode the list, use a duplicate filter to set the value of anything coming out of the duplicate port to blank then rebuild the list?

 

Edit: just noticed you need the first element to be replaced with a blank rather than the second which makes it a bit more complicated

Userlevel 1
Badge +21

Are your duplicates always going to be adjacent to each other?

Badge +1

Are your duplicates always going to be adjacent to each other?

Not necessarily. I think at this point, I would just settle for no duplicates/right number of elements.

Am trying your suggestion - ListExploder, DuplicateFilter, using AttributeManager on the duplicate port to set value to ' ' and plugging it all into ListConcatenator. Currently, though, LC cannot see any list attributes...

Userlevel 1
Badge +21

remove_duplicates_from_list.fmw

Userlevel 1
Badge +21

Not necessarily. I think at this point, I would just settle for no duplicates/right number of elements.

Am trying your suggestion - ListExploder, DuplicateFilter, using AttributeManager on the duplicate port to set value to ' ' and plugging it all into ListConcatenator. Currently, though, LC cannot see any list attributes...

After exploding you no longer have a list, you need to rebuild the list before concatenating

Badge +1

After exploding you no longer have a list, you need to rebuild the list before concatenating

Yes, of course. Where's the facepalm emoji on this thing...?

Userlevel 1
Badge +21

If i could replace 8 transformers with one python caller though I'd probably go down that route

import fme
import fmeobjects

def replace_duplicates_with_blanks(feature):
    text = feature.getAttribute('text')
    list = text.split("|")
    list.reverse()
    newlist=[]
    for i, val in enumerate(list):
        if val in newlist:
            newlist.append("")
        else:
            newlist.append(val)
    newlist.reverse()
    separator = '|'
    newtext = separator.join(newlist)
    feature.setAttribute('newtext',newtext)

Reply