Skip to main content
Question

CSV reader fail to read negative integer as int. Issues with encoding


so_much_more
Supporter
Forum|alt.badge.img+6

I struggle to make FME read csv files with correct encoding and datatype.

I receive csv files in UTF-8 that includes non ASCII characters.

FME think its win-1252 encoding and this interpret signed integer values like -27 as −27 and thus make it a character datatype.

To fix the encoding i have to add the csv reader to the workbench and then change en Character encoding to UTF-8, then it works. But at that time the datatype is already set and I cannot change the varchar type column to be int32. It cause issues becasue now I cannot write to my database which expect int and not varchar...

So how can I change the datatype of multiple fields after the reader has been added to the workbench? Alternatively (and better), define the CSV reader to be using UTF-8 before the CSV file has been scanned for its content

 

 

 

3 replies

geomancer
Evangelist
Forum|alt.badge.img+51
  • Evangelist
  • October 31, 2023

Workaround: in the Text File Reader you can set the Chararter Encoding (as part of the Parameters).

So a possible solution is to read the CSV as a Text File, and split the attributes afterwards.


david_r
Celebrity
  • October 31, 2023

You can go into Tools / FME Options and activate this setting, which will allow you to modify the reader data types:

imageThis way it should be possible to change from string to int32 in your CSV reader definition.


so_much_more
Supporter
Forum|alt.badge.img+6
  • Author
  • Supporter
  • October 31, 2023
david_r wrote:

You can go into Tools / FME Options and activate this setting, which will allow you to modify the reader data types:

imageThis way it should be possible to change from string to int32 in your CSV reader definition.

Thank you! That was an option well worth knowing. That solves the issues, thanks 👍


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings