Question
Splitting large text files
+1Hi, I am a new FME user and am hoping you can help me. I have river flow data from several gauging stations contained within one text file. I need to split the data based on gauging station - so that the data from each station is contained in its own Access db table. The station number is contained in header information at the start of each chunk of data, but is not within each data row. So I need a transformer to write the data until it finds the next station number, and then start writing to a new destination file and so on until the end of the file. What transformer should I use? Thanks
Hi,
you can use the Fanout Dataset properties of the writer, have a look at the documentation for examples. No need for any transformers :-)
You can also find more info here.
David
+1Thanks for your answers. I will have to take a step back and ask how I get FME to recognise that the data in my text files is tab separated? When I write the data to a csv or excel file all of one row appears in a single cell.
Thanks,
Abby
+1Have sorted it. Read the text file in as a CSV but set the field separator to tab. Then used a Tester, VariableSetter and VariableRetriever as suggested by Mark, and then a fanout. Works fine, thanks for your help.
Abby
Reply
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.