Skip to main content
Question

Merging several CSV files with different Schema

  • November 23, 2020
  • 2 replies
  • 144 views
G
Forum|alt.badge.img

I have a FME 2019 Desktop workflow that reads three CSV files and write all input data (points) to only one FeatureClass in the output fileGeodatabase.

Every CSV file has a different schema, i.e.: every CSV file has several fields in common but most of them are different.

This is the workflow:csv2gdbEvery month I will have to run this workflow with a different set of CSV files with probably will have different schema than this ones, and also could be more than three CSV files.

So, how can I read automatically the schema of CSV files ? Thank you !

 

2 replies

G
Forum|alt.badge.img
  • gerard
  • Author
  • November 24, 2020

I have found the appropiate combination of parameters, and the answer is very simple:

  1. The CSV reader select all *.csv files in a directory, with Single Merged Feature Type and Automatic read of Schema
  2. The File Geodb writer, has to define FeatureClass or Table Definition as Automatic .... (not Dynamic, as I thought)

workflow2 

Now I have the output structure that i need but I have to solve the problem of generate points form CSV input data. If I define ReaderSchema as Manual I can assign several fields to (x,y,z) coordinates, but I have defined that option Automatic ....

 

One possible solution is do it in 2 steps:

  1. Generate a merged CSV with the combined structure of fields,
  2. Generate a GDB with this merged CSV reading manually the Schema

 

Any idea for solving that in one step ? Thank you

G
Forum|alt.badge.img
  • gerard
  • Author
  • November 24, 2020
gerard wrote:

I have found the appropiate combination of parameters, and the answer is very simple:

  1. The CSV reader select all *.csv files in a directory, with Single Merged Feature Type and Automatic read of Schema
  2. The File Geodb writer, has to define FeatureClass or Table Definition as Automatic .... (not Dynamic, as I thought)

workflow2 

Now I have the output structure that i need but I have to solve the problem of generate points form CSV input data. If I define ReaderSchema as Manual I can assign several fields to (x,y,z) coordinates, but I have defined that option Automatic ....

 

One possible solution is do it in 2 steps:

  1. Generate a merged CSV with the combined structure of fields,
  2. Generate a GDB with this merged CSV reading manually the Schema

 

Any idea for solving that in one step ? Thank you

Ok, VertexCreator is the solution.

Reply


Helpful Members This Week

Community Stats

31,340
Posts
118,850
Replies
38,929
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.