Skip to main content
Question

Reader Dynamic CSV Schema Changes

  • March 13, 2020
  • 1 reply
  • 131 views
J

I've been looking at all the various options people have for Dynamic Schema changes in a CSV reader. But the only thing that consistently functions seems to be the "Update Reader" Menu option.

Scenario: I have a single CSV where the schema gets added to on a regular basis

Week 1: 'col1, col2, col3, 'week 1'

Week 2: 'col1, col2, col3, 'week 1', 'week 2'

Week 3: 'col1, col2, col3, 'week 1', 'week 2', 'week 3' etc.

I need to be able to read in the CSV and output to a DBMS table, pulling in the new column each week.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

1 reply

daveatsafe
Safer
Forum|alt.badge.img+20
  • daveatsafeSafer
  • Safer
  • March 13, 2020

Please use a FeatureReader transformer instead of the CSV reader.

Set Format type to CSV and pick your CSV file as the source dataset. Set Output - Output Ports to Single Output Port. Leave all other settings as the default.

 

If you are writing out to a new table each week, connect both the <Schema> and <Generic> ports to the database writer. In the output Feature Type Parameters, enable Dynamic Schema Definition and set Schema Sources to "Schema from Schema Feature".

When the workspace is run, the FeatureReader will scan the CSV file to get the schema, then supply that to the database writer on the Schema feature. The dynamic mode on the writer will create the new table using the schema feature definition. The data features will be output after the Schema feature, from the Generic port.

Community Stats

32,180
Posts
122,169
Replies
39,855
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.