Skip to main content
  • EN

I struggle to make FME read csv files with correct encoding and datatype.

I receive csv files in UTF-8 that includes non ASCII characters.

FME think its win-1252 encoding and this interpret signed integer values like -27 as −27 and thus make it a character datatype.

To fix the encoding i have to add the csv reader to the workbench and then change en Character encoding to UTF-8, then it works. But at that time the datatype is already set and I cannot change the varchar type column to be int32. It cause issues becasue now I cannot write to my database which expect int and not varchar...

So how can I change the datatype of multiple fields after the reader has been added to the workbench? Alternatively (and better), define the CSV reader to be using UTF-8 before the CSV file has been scanned for its content

 

 

 

Workaround: in the Text File Reader you can set the Chararter Encoding (as part of the Parameters).

So a possible solution is to read the CSV as a Text File, and split the attributes afterwards.

You can go into Tools / FME Options and activate this setting, which will allow you to modify the reader data types:

imageThis way it should be possible to change from string to int32 in your CSV reader definition.

You can go into Tools / FME Options and activate this setting, which will allow you to modify the reader data types:

imageThis way it should be possible to change from string to int32 in your CSV reader definition.

Thank you! That was an option well worth knowing. That solves the issues, thanks 👍

Reply


Community Stats

30,108
Posts
114,295
Replies
37,745
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.