Skip to main content
Question

Merging several CSV files with different Schema

  • November 23, 2020
  • 2 replies
  • 183 views
gerard
Participant
Forum|alt.badge.img

I have a FME 2019 Desktop workflow that reads three CSV files and write all input data (points) to only one FeatureClass in the output fileGeodatabase.

Every CSV file has a different schema, i.e.: every CSV file has several fields in common but most of them are different.

This is the workflow:csv2gdbEvery month I will have to run this workflow with a different set of CSV files with probably will have different schema than this ones, and also could be more than three CSV files.

So, how can I read automatically the schema of CSV files ? Thank you !

 

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

gerard
Participant
Forum|alt.badge.img
  • gerardParticipant
  • Author
  • Participant
  • November 24, 2020

I have found the appropiate combination of parameters, and the answer is very simple:

  1. The CSV reader select all *.csv files in a directory, with Single Merged Feature Type and Automatic read of Schema
  2. The File Geodb writer, has to define FeatureClass or Table Definition as Automatic .... (not Dynamic, as I thought)

workflow2 

Now I have the output structure that i need but I have to solve the problem of generate points form CSV input data. If I define ReaderSchema as Manual I can assign several fields to (x,y,z) coordinates, but I have defined that option Automatic ....

 

One possible solution is do it in 2 steps:

  1. Generate a merged CSV with the combined structure of fields,
  2. Generate a GDB with this merged CSV reading manually the Schema

 

Any idea for solving that in one step ? Thank you

gerard
Participant
Forum|alt.badge.img
  • gerardParticipant
  • Author
  • Participant
  • November 24, 2020

I have found the appropiate combination of parameters, and the answer is very simple:

  1. The CSV reader select all *.csv files in a directory, with Single Merged Feature Type and Automatic read of Schema
  2. The File Geodb writer, has to define FeatureClass or Table Definition as Automatic .... (not Dynamic, as I thought)

workflow2 

Now I have the output structure that i need but I have to solve the problem of generate points form CSV input data. If I define ReaderSchema as Manual I can assign several fields to (x,y,z) coordinates, but I have defined that option Automatic ....

 

One possible solution is do it in 2 steps:

  1. Generate a merged CSV with the combined structure of fields,
  2. Generate a GDB with this merged CSV reading manually the Schema

 

Any idea for solving that in one step ? Thank you

Ok, VertexCreator is the solution.

Community Stats

32,154
Posts
122,047
Replies
39,812
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.