The attached data has 164028 rows but only 134483 row are actually read. I can import using Excel and this has the correct amount of data but if I use FME I get less, anyway I can resolve this? I have also attached the parameters I used. Cheers
Solved
Why would fme csv reader be missing rows
Best answer by redgeographics
It appears the file is not set up uniformly. If I run it with the default settings, which scans the first 10000 rows for the schema, I get a lot of warnings. This is due to a " character in one of the values.
Setting the Field Qualifier Character and Field Qualifier Escape Character parameters to empty solves the issue and all rows are being read.
+50- Celebrity
- Best Answer
It appears the file is not set up uniformly. If I run it with the default settings, which scans the first 10000 rows for the schema, I get a lot of warnings. This is due to a " character in one of the values.
Setting the Field Qualifier Character and Field Qualifier Escape Character parameters to empty solves the issue and all rows are being read.
+44- Safer
I'd just chime in to mention that there is a CSVValidator custom transformer on the FME Hub that helps for this sort of issue. I ran your data through this and it found the issues you were experiencing (qualifier characters, etc). The only problem is that it assumed a comma delimiter. I'll see if I can change it to be user-defined.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.