Skip to main content
Question

Format Text File where Column Names are Repeated on Each Row


tim.albert.vaa
Contributor
Forum|alt.badge.img+9

I have a text file that looks like this:

"CommonCode":"EI","IATA":"EI","ICAO":"EIN","CarrierName":"Aer Lingus"
"CommonCode":"AM","IATA":"AM","ICAO":"AMX","CarrierName":"Aeromexico"

I want it to look like this:

CommonCode IATA ICAO CarrierName
EI EI EIN Aer Lingus
AM AM AMX Aeromexico


I want it to create the column names dynamically from the text file as the column names will change and increase or decrease in number.

In R there appears to be a command called:  “unsparsify”   Which according to this post seems to do it.https://datascience.stackexchange.com/questions/55879/file-format-where-column-names-are-repeated-on-each-row

I can do it using a combination of AttributeSplitters, ListExploders, ListIndexers, AttributePivoters and AttributeExposers.   But I can’t seem to get away from having to Explicitly input the Attribute names in the AttributeExposer and I need it to be dynamic.

Any ideas?

7 replies

DanAtSafe
Safer
Forum|alt.badge.img+17
  • Safer
  • March 21, 2025

Hi Tim, Did you try a SchemaScanner? You don’t need to expose attributes for dynamic writing. 

I got this written out to Excel: 

Note that the fields are sorted alphabetically.


mark2atsafe
Safer
Forum|alt.badge.img+43

Nice ​@DanAtSafe - are you going to add this to the FME Hub as a new transformer?!


crutledge
Enthusiast
Forum|alt.badge.img+27
  • Enthusiast
  • March 21, 2025
mark2atsafe wrote:

Nice ​@DanAtSafe - are you going to add this to the FME Hub as a new transformer?!

I vote yes! This is an elegant solution. Something I will definitely use down the road (or run into and have to solve at that point :D)


tim.albert.vaa
Contributor
Forum|alt.badge.img+9

Hi Dan,

Thank you for the excellent solution—it worked perfectly for my needs.

The real “aha” moment for me was your use of the AttributeManager to set the field name from List value (0) and the data from List value (1). That approach had never occurred to me, and I likely wouldn’t have thought of it on my own.

I also noticed that the dropdown list for “Output Attribute” doesn’t show Lists as an option. If there’s a way to include Lists in that selection, it might help others realize that list values can be used as output attribute names. That could make this powerful technique more discoverable.

Thanks again!


ebygomm
Influencer
Forum|alt.badge.img+31
  • Influencer
  • March 26, 2025

Since it’s effectively json, you can just use a json flattener

 


tim.albert.vaa
Contributor
Forum|alt.badge.img+9

Hi Ebygomm,

Thanks so much for the suggestion!

Yes, that works—as long as I wrap each line in curly brackets { }. Thankfully, that's easy to do using the StringConcatenator.

Without the curly brackets, the JSONFlattener rejects the input. Not being very familiar with JSON, this was another option I hadn’t considered.

Really appreciate the help from everyone—thank you!


ebygomm
Influencer
Forum|alt.badge.img+31
  • Influencer
  • March 26, 2025

You can add the brackets directly in the JSON flattener if you use the text editor when setting the JSON document as in the screenshot. 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings