Skip to main content
  • EN

I've been looking at all the various options people have for Dynamic Schema changes in a CSV reader. But the only thing that consistently functions seems to be the "Update Reader" Menu option.

Scenario: I have a single CSV where the schema gets added to on a regular basis

Week 1: 'col1, col2, col3, 'week 1'

Week 2: 'col1, col2, col3, 'week 1', 'week 2'

Week 3: 'col1, col2, col3, 'week 1', 'week 2', 'week 3' etc.

I need to be able to read in the CSV and output to a DBMS table, pulling in the new column each week.

Please use a FeatureReader transformer instead of the CSV reader.

Set Format type to CSV and pick your CSV file as the source dataset. Set Output - Output Ports to Single Output Port. Leave all other settings as the default.

 

If you are writing out to a new table each week, connect both the <Schema> and <Generic> ports to the database writer. In the output Feature Type Parameters, enable Dynamic Schema Definition and set Schema Sources to "Schema from Schema Feature".

When the workspace is run, the FeatureReader will scan the CSV file to get the schema, then supply that to the database writer on the Schema feature. The dynamic mode on the writer will create the new table using the schema feature definition. The data features will be output after the Schema feature, from the Generic port.

Reply


Community Stats

29,733
Posts
112,967
Replies
37,380
Members
Terms & ConditionsCookie settings