Skip to main content
Solved

Dynamically Read Multiple CSV Schema - Organize - Export as Standard CSV Structure

  • Mar 28, 2022 4:57 PM UTC
  • 3 replies
  • 297 views

canerakin
Contributor
Forum|alt.badge.img+6

Goal: 

I am trying to standardize the raw CSV files. The output file columns should look like below.

Date	Time	Level	Temperature	Elevation

Background Info: 

Each file might have multiple tabs. I need to only process the sheets that named as 20* (2011-2012-2019-2020 etc). Sometimes there is header information in A column in some of the sheets. Thus the table header starts at different row number.

Issue: 

I am using FeatureReader to detect the schema dynamically. If the table starts with header information, the transformer reads the data as a new table. Thus it creates new column names which turns into a larger issue.

Normally there are over 200 files with multiple tabs but I've prepared a sample data and included my workbench. Looking for help.

Best answer by nielsgerrits

For this reason I prefer to use the Generic Reader to dynamically read Excel files. Then you get col_1, col_2, etc... which you can rename yourself. Attached sample with your data.

View original
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

3 replies

nielsgerrits
VIP
Forum|alt.badge.img+67
  • Best Answer
  • Mar 29, 2022 10:17 AM UTC

For this reason I prefer to use the Generic Reader to dynamically read Excel files. Then you get col_1, col_2, etc... which you can rename yourself. Attached sample with your data.


canerakin
Contributor
Forum|alt.badge.img+6
  • Author
  • Contributor
  • Mar 29, 2022 1:34 PM UTC

For this reason I prefer to use the Generic Reader to dynamically read Excel files. Then you get col_1, col_2, etc... which you can rename yourself. Attached sample with your data.

Hi @nielsgerrits​ , I tried so many things but never thought about Generic reader. It definitely gave me a better direction, thanks a lot!


nielsgerrits
VIP
Forum|alt.badge.img+67

Hi @nielsgerrits​ , I tried so many things but never thought about Generic reader. It definitely gave me a better direction, thanks a lot!

Happy I could help. :)