The attached data has 164028 rows but only 134483 row are actually read. I can import using Excel and this has the correct amount of data but if I use FME I get less, anyway I can resolve this? I have also attached the parameters I used. Cheers
Solved
Why would fme csv reader be missing rows
Best answer by redgeographics
It appears the file is not set up uniformly. If I run it with the default settings, which scans the first 10000 rows for the schema, I get a lot of warnings. This is due to a " character in one of the values.
Setting the Field Qualifier Character and Field Qualifier Escape Character parameters to empty solves the issue and all rows are being read.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+59- Celebrity
- 3698 replies
- Best Answer
It appears the file is not set up uniformly. If I run it with the default settings, which scans the first 10000 rows for the schema, I get a lot of warnings. This is due to a " character in one of the values.
Setting the Field Qualifier Character and Field Qualifier Escape Character parameters to empty solves the issue and all rows are being read.
+56- Safer
- 2554 replies
I'd just chime in to mention that there is a CSVValidator custom transformer on the FME Hub that helps for this sort of issue. I ran your data through this and it found the issues you were experiencing (qualifier characters, etc). The only problem is that it assumed a comma delimiter. I'll see if I can change it to be user-defined.