Question
Splitting large text files
+2Hi, I am a new FME user and am hoping you can help me. I have river flow data from several gauging stations contained within one text file. I need to split the data based on gauging station - so that the data from each station is contained in its own Access db table. The station number is contained in header information at the start of each chunk of data, but is not within each data row. So I need a transformer to write the data until it finds the next station number, and then start writing to a new destination file and so on until the end of the file. What transformer should I use? Thanks
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
Hi,
you can use the Fanout Dataset properties of the writer, have a look at the documentation for examples. No need for any transformers :-)
You can also find more info here.
David
+2Thanks for your answers. I will have to take a step back and ask how I get FME to recognise that the data in my text files is tab separated? When I write the data to a csv or excel file all of one row appears in a single cell.
Thanks,
Abby
+2Have sorted it. Read the text file in as a CSV but set the field separator to tab. Then used a Tester, VariableSetter and VariableRetriever as suggested by Mark, and then a fanout. Works fine, thanks for your help.
Abby