Hello I am brand new FME this week and am intrigued by it's non spatial data integration possibilities.
I have a variety of csv files to process. I know explicitly how the header of each file is setup so would like to be able to test for certain conditions i.e. 1st two rows blank and (row 3 column 1 = 'Datetime') is Filetype A if successful this will go forward to relevant workspace runner for that filetype so I can do the required processing.
Is it possible to test the structure of CSV before determining how to process it?
Thanks and kind regards
Page 1 / 1
Hi
You could analyze the contents of the files using the Text File reader, which will return one feature per text line. You could use a Counter to keep track of which line whatever text appears on.
David
You could analyze the contents of the files using the Text File reader, which will return one feature per text line. You could use a Counter to keep track of which line whatever text appears on.
David
You can analyse and parse any texfile. Adn then decide what to do with it.
Useing textfile-reader you get ,amongst others, attributes text_line and line_number.
Main tools would be of course regular expressions, variableSetter and Retriever. Possibly some stringoperations.
A csv is, afaik, already ordered though, all you would need to do is read it. Set settings in reader parameter.
Useing textfile-reader you get ,amongst others, attributes text_line and line_number.
Main tools would be of course regular expressions, variableSetter and Retriever. Possibly some stringoperations.
A csv is, afaik, already ordered though, all you would need to do is read it. Set settings in reader parameter.
Hi
You could analyze the contents of the files using the Text File reader, which will return one feature per text line. You could use a Counter to keep track of which line whatever text appears on.
David
You could analyze the contents of the files using the Text File reader, which will return one feature per text line. You could use a Counter to keep track of which line whatever text appears on.
David
Thanks @david_r I will give it a try and report back
You can analyse and parse any texfile. Adn then decide what to do with it.
Useing textfile-reader you get ,amongst others, attributes text_line and line_number.
Main tools would be of course regular expressions, variableSetter and Retriever. Possibly some stringoperations.
A csv is, afaik, already ordered though, all you would need to do is read it. Set settings in reader parameter.
Useing textfile-reader you get ,amongst others, attributes text_line and line_number.
Main tools would be of course regular expressions, variableSetter and Retriever. Possibly some stringoperations.
A csv is, afaik, already ordered though, all you would need to do is read it. Set settings in reader parameter.
Thanks @gio I will also give this a try. I will only need to analyse the first 12 lines of any file max so could do with any hints to limiting how much of the file it parses. Thanks again
On the CSV reader you can set the Max Features to read to 12, as there is one feature per line.
Thanks everyone for your pointers. I have got it done I don't know if it is the most elegant but it works.
I have a series of test for each file type if they pass I let one Feature through and create an attribute 'Filetype' and give it the relevant number. This is then appended to a csv file detailing filename and filetype.
I am really enjoying using this tool goodness knows how long this would have taken in VBA
I have a series of test for each file type if they pass I let one Feature through and create an attribute 'Filetype' and give it the relevant number. This is then appended to a csv file detailing filename and filetype.
I am really enjoying using this tool goodness knows how long this would have taken in VBA
Thanks everyone for your pointers. I have got it done I don't know if it is the most elegant but it works.
I have a series of test for each file type if they pass I let one Feature through and create an attribute 'Filetype' and give it the relevant number. This is then appended to a csv file detailing filename and filetype.
I am really enjoying using this tool goodness knows how long this would have taken in VBA ;-)
I have a series of test for each file type if they pass I let one Feature through and create an attribute 'Filetype' and give it the relevant number. This is then appended to a csv file detailing filename and filetype.
I am really enjoying using this tool goodness knows how long this would have taken in VBA ;-)
Good job @tomcrumbleholme -- my only suggestion would be to consider the TestFilter instead of the cascading Testers -- that might clean it up a bit. Glad you're enjoying FME. Warning -- it can be addictive!
Good job @tomcrumbleholme -- my only suggestion would be to consider the TestFilter instead of the cascading Testers -- that might clean it up a bit. Glad you're enjoying FME. Warning -- it can be addictive!
Thanks Dale, so I can just have series of else if statements in the filter?
Thanks Dale, so I can just have series of else if statements in the filter?
Yes, each line in the TestFilter is equivalent to an else if statement. For more info on the Tester vs TestFilter, check out: https://blog.safe.com/2013/03/fmeevangelist113/ and https://blog.safe.com/2015/05/fmeevangelist133/
Update: Even more elegant use AttributeCreator then set to conditional value. Will upload cleaned up workbench later
Update: Even more elegant use AttributeCreator then set to conditional value. Will upload cleaned up workbench later
Well of course! Don't know why we didn't think of that ourselves :-)
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.