setting up user parameters for readers

So if I understand your question correctly, for each job/translation you are combining the data of a bunch of CSV files (of one specific schema) together into one large CSV file. But the batch of CSV files for a different job/translation may have have a new (uniform) schema?

I guess it's best to use the dynamic reader and writer functionality of FME. You can find more information about dynamic workflows in this article from Safe.

There are a couple of approaches you can follow. Personally I prefer using FeatureReaders and FeatureWriters, as they provide a bit more flexibility, and then your workspace could look like this;

But if you choose to use regular Readers and Writers, your workspace could become even more compact by cloning the schema of the Reader to the Writer. Then it could look as simple as this;

Hope it can help.

Hi, yes that is exactly what I am looking to do. I tried the dynamic readers approach and it still retains the schema from the first batch of csvs and I don't understand why as this is precisely what this approach is set up to do. Not sure if there is something I'm doing wrong. I've attempted it using both the "generate workspace" approach and by setting the readers and writers as dynamic and neither has worked. my workflow looks like this but when I select a new set of csvs and run it retains the original schema Not sure why this is happening

Hi, @ocarey​ ,

You approach seems similar to the WS 'combine_CSV_files_option_2.fmw' that is contained in the zipfile I attached in my previous post. Can you maybe check if that workspace would work for you (I hope you can open the recent fmw, I'm using FME 2021.2.2.0 )?

I think your issue could be related to the naming of the feature types and schema features. When Adding the CSV Reader, what setting did you use for the Feature Type Name(s)? The default 'From Format Name', or did you switch to 'From File Name(s)'?

Similarly, what kind of settings did you use for the writer feature types?

Did you maybe change the CSV File name parameter to a new (user parameter) value?

For example, if you use the default Reader FT name option 'From Format Name', all CSV feature types will be named 'CSV'. But then if you change the CSV File name at the writer, and leave the Schema Sources and Schema Definition Name to the default values, you will get a mismatch. In this case your writer would be expecting a schema with your newly set/configured CSV file name (Default for Schema Definition Name is 'Default from CSV File name above'), but your reader is still using the feature type name 'CSV'.

However, in this case you wouldn't get any features to be written, where you mention that the features are being written but with the original schema.

Nevertheless I think these Reader/Writer FT settings could be the cause of undesired translation behavior. Maybe you can add some screenshots to show how (the feature types of) your Dynamic Reader and Writer are configured? Or even better, maybe you can add the workspace to this case so I can have a look at what might be going wrong?