Skip to main content
  • EN

Hi there,

if I have a dynamic filename within a csv or feature writer, e.g. "@Value(LINK_DIRECTION)_@Value(SIMULATION)_@Value(TECHNOLOGY)_@Value(BAND)_@Value(CARRIER)_@Value(TYPE)_@Value(SERVER)-importfile" it takes about 25 seconds to write one million files. If I write simply to test.csv, it takes about 2 seconds.

Both variants writes only one file. I tried also to group by within the featurewriter with option "Complete Groups when Group Changes". All records are within the same group.

User Attribute Definition is in all cases set to "Automatic".

What did I wrong?

What's the best way to just write into a csv file, with various attributes in the filename?

Thank you for any help!

I'm using FME 2022.2

I think using parameters is quicker as these cannot change during runtime.

Thanks for your answer.

Not sure, if I understood you right: parameters are set at the beginning, no? But i have to set the filename regarding values in the attributes of the data, which could change.

In total, in the end I should have more than 30 csv files grouped by those attributes, everyone contains then millions of records.

Or do you mean to define parameters and change them within the pipeline somehow?

I also think, that this pipeline could be a great oppurtunity for parallelization, as i have a featureReader and featureWriter.

Reply


Community Stats

30,108
Posts
114,295
Replies
37,744
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.