Anyone have a good way to drop a duplicate attribute name?
Happy 2025!
I’m processing some OpenStreetMap PBF data and have found a duplicate attribute name, hard to believe but I think an artifact of OSM’s XML ancestry, two attributes differ only in string case, like my_attribute and My_Attibute. My processing changes all attributes to lowercase so I have two fields my_attribute. I tried a ListDuplicateRemover on the Schema output of a SchemaScanner but I get an invalid schema definition. Does anyone with knowledge of FME internals have a tip for dropping one of the duplicate attributes? Thanks.
Page 1 / 1
Wow, seems strange - and you’re quite sure that these attributes don’t have anything like a space at the end? How are you changing the case? I’ve not been able to reproduce the effect of having two attributes with the same name if I use the BulkAttributeRenamer.
if you you run everything through a BulkAttributeRenamer maybe that will “fix” the issue?
Hi Matt, I am using a BulkAttributeRenamer to change all attributes to lowercase. I read PBF then immediately make field names geodatabase-valid by replacing unsupported characters as well, busting to lowercase is part of that. i’ll go looking for stray non-printing characters - thanks.
As @virtualcitymatt says, I think it’s unlikely that the attributes differ by case only. If they only differ by case, you should see the top behaviour in the BulkAttributeRenamer. The bottom example looks the same, but there’s a non-breaking space on the end of one of the attributes.
I think i may have a python example somewhere that identifies attributes that ‘look’ like they are the same but aren’t in reality due to non-printing characters. I had some fun with some change detection once
This might also work
Thanks all - I think I accidentally found ebygomm’s case & fix in my processing, when I didn’t have a BulkAttributeRenamer in my workspace the problem presented but when I do it works fine.