Skip to main content
Question

Reader Dynamic CSV Schema Changes

  • March 13, 2020
  • 1 reply
  • 85 views
J

I've been looking at all the various options people have for Dynamic Schema changes in a CSV reader. But the only thing that consistently functions seems to be the "Update Reader" Menu option.

Scenario: I have a single CSV where the schema gets added to on a regular basis

Week 1: 'col1, col2, col3, 'week 1'

Week 2: 'col1, col2, col3, 'week 1', 'week 2'

Week 3: 'col1, col2, col3, 'week 1', 'week 2', 'week 3' etc.

I need to be able to read in the CSV and output to a DBMS table, pulling in the new column each week.

1 reply

daveatsafe
Safer
Forum|alt.badge.img+19
  • daveatsafeSafer
  • Safer
  • March 13, 2020

Please use a FeatureReader transformer instead of the CSV reader.

Set Format type to CSV and pick your CSV file as the source dataset. Set Output - Output Ports to Single Output Port. Leave all other settings as the default.

 

If you are writing out to a new table each week, connect both the <Schema> and <Generic> ports to the database writer. In the output Feature Type Parameters, enable Dynamic Schema Definition and set Schema Sources to "Schema from Schema Feature".

When the workspace is run, the FeatureReader will scan the CSV file to get the schema, then supply that to the database writer on the Schema feature. The dynamic mode on the writer will create the new table using the schema feature definition. The data features will be output after the Schema feature, from the Generic port.

Reply


Helpful Members This Week

Community Stats

30,942
Posts
117,252
Replies
38,742
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.