Skip to main content
Solved

Dynamically Read Multiple CSV Schema - Organize - Export as Standard CSV Structure


canerakin
Contributor
Forum|alt.badge.img+5

Goal: 

I am trying to standardize the raw CSV files. The output file columns should look like below.

Date	Time	Level	Temperature	Elevation

Background Info: 

Each file might have multiple tabs. I need to only process the sheets that named as 20* (2011-2012-2019-2020 etc). Sometimes there is header information in A column in some of the sheets. Thus the table header starts at different row number.

Issue: 

I am using FeatureReader to detect the schema dynamically. If the table starts with header information, the transformer reads the data as a new table. Thus it creates new column names which turns into a larger issue.

Normally there are over 200 files with multiple tabs but I've prepared a sample data and included my workbench. Looking for help.

Best answer by nielsgerrits

For this reason I prefer to use the Generic Reader to dynamically read Excel files. Then you get col_1, col_2, etc... which you can rename yourself. Attached sample with your data.

View original
Did this help you find an answer to your question?

3 replies

nielsgerrits
VIP
Forum|alt.badge.img+54
  • Best Answer
  • March 29, 2022

For this reason I prefer to use the Generic Reader to dynamically read Excel files. Then you get col_1, col_2, etc... which you can rename yourself. Attached sample with your data.


canerakin
Contributor
Forum|alt.badge.img+5
  • Author
  • Contributor
  • March 29, 2022
nielsgerrits wrote:

For this reason I prefer to use the Generic Reader to dynamically read Excel files. Then you get col_1, col_2, etc... which you can rename yourself. Attached sample with your data.

Hi @nielsgerrits​ , I tried so many things but never thought about Generic reader. It definitely gave me a better direction, thanks a lot!


nielsgerrits
VIP
Forum|alt.badge.img+54
canerakin wrote:

Hi @nielsgerrits​ , I tried so many things but never thought about Generic reader. It definitely gave me a better direction, thanks a lot!

Happy I could help. :)


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings