Skip to main content
Solved

Anyone have a good way to drop a duplicate attribute name?

  • January 13, 2025
  • 5 replies
  • 44 views

bruceharold
Contributor
Forum|alt.badge.img+17

Happy 2025!

I’m processing some OpenStreetMap PBF data and have found a duplicate attribute name, hard to believe but I think an artifact of OSM’s XML ancestry, two attributes differ only in string case, like my_attribute and My_Attibute.  My processing changes all attributes to lowercase so I have two fields my_attribute.  I tried a ListDuplicateRemover on the Schema output of a SchemaScanner but I get an invalid schema definition.  Does anyone with knowledge of FME internals have a tip for dropping one of the duplicate attributes?  Thanks.

Best answer by ebygomm

This might also work

 

View original
Did this help you find an answer to your question?

virtualcitymatt
Celebrity
Forum|alt.badge.img+35

Wow, seems strange - and you’re quite sure that these attributes don’t have anything like a space at the end? How are you changing the case? I’ve not been able to reproduce the effect of having two attributes with the same name if I use the BulkAttributeRenamer.

if you you run everything through a BulkAttributeRenamer maybe that will “fix” the issue?

  


bruceharold
Contributor
Forum|alt.badge.img+17

Hi Matt, I am using a BulkAttributeRenamer to change all attributes to lowercase.  I read PBF then immediately make field names geodatabase-valid by replacing unsupported characters as well, busting to lowercase is part of that.  i’ll go looking for stray non-printing characters - thanks.


ebygomm
Influencer
Forum|alt.badge.img+32
  • Influencer
  • January 14, 2025

As ​@virtualcitymatt says, I think it’s unlikely that the attributes differ by case only. If they only differ by case, you should see the top behaviour in the BulkAttributeRenamer. The bottom example looks the same, but there’s a non-breaking space on the end of one of the attributes.

I think i may have a python example somewhere that identifies attributes that ‘look’ like they are the same but aren’t in reality due to non-printing characters. I had some fun with some change detection once


ebygomm
Influencer
Forum|alt.badge.img+32
  • Influencer
  • Best Answer
  • January 14, 2025

This might also work

 


bruceharold
Contributor
Forum|alt.badge.img+17

Thanks all - I think I accidentally found ebygomm’s case & fix in my processing, when I didn’t have a BulkAttributeRenamer in my workspace the problem presented but when I do it works fine.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings