Solved

Retain Chinese characters with SQL writer

  • 29 November 2016
  • 7 replies
  • 8 views

Badge

Hi,

I have an output that has Chinese characters but when I write back to sql (generate a new table

MSSQL_ADO) the characters are lost. The data comes from SQL and FME handles the characters correctly up to the point the writer is used any suggestions? Thanks.

icon

Best answer by lenaatsafe 29 November 2016, 19:52

View original

7 replies

Userlevel 2
Badge +17
Firstly check if it is an issue that is specific to Chinese characters. If the field stores only ASCII characters, those can be written into the destination table correctly?

 

Badge

works fine on English characters as I have processed various countries?

Userlevel 2
Badge +17

works fine on English characters as I have processed various countries?

OK. There might be an issue on FME internationalization here. I would recommend you to contact Safe support if you believe it's an issue specific to Chinese characters.

 

In fact, there are potential issues on internationalization. I've experienced many issues on Japanese characters, reported everything to Safe support, and many have been resolved but there still are outstanding issues unfortunately.

 

Userlevel 4
Badge +25

Is it attribute names or values? Names are always a bit harder to do, and I would recommend updating to the latest FME version if necessary (you don't say which you are using).

As Takashi says, this is probably one to take up with our support team. I did a quick look for existing reports and can't find any, so either it should work, or no-one has ever asked for this before. I suspect it should work, but the ADO writer might be different. Could you try the JDBC non-spatial writer to see if the result is the same? Or maybe even try the Spatial version, just ignoring geometry.

Sorry I can't be much more help but I hope either a different writer or the support team will be able to get you going

Badge

Hi @denby

you have Chinese characters in attribute values (not in attribute names), am I correct?

Is your SQL Server default language English? You can check this through Microsoft SQL Server Management Studio > Server Properties > Advanced > Default Language. You can set the default language to Chinese (either Simplified or Traditional) and this will probably fix the problem. However, this is not the easiest option.

You are probably writing the Chinese values into char or varchar columns. Both - char and varchar - support values in server default language encoding (which should be Win-1252 in case of English). Instead of char/varchar you can use nchar/nvarchar that by default store Unicode values - this should solve the problem without affecting all your SQL Server databases/users.

Badge

Hi @denby

you have Chinese characters in attribute values (not in attribute names), am I correct?

Is your SQL Server default language English? You can check this through Microsoft SQL Server Management Studio > Server Properties > Advanced > Default Language. You can set the default language to Chinese (either Simplified or Traditional) and this will probably fix the problem. However, this is not the easiest option.

You are probably writing the Chinese values into char or varchar columns. Both - char and varchar - support values in server default language encoding (which should be Win-1252 in case of English). Instead of char/varchar you can use nchar/nvarchar that by default store Unicode values - this should solve the problem without affecting all your SQL Server databases/users.

Great thankyou, its fixed using nvarchar instead of varchar

Userlevel 4
Badge +25

Hi @denby

you have Chinese characters in attribute values (not in attribute names), am I correct?

Is your SQL Server default language English? You can check this through Microsoft SQL Server Management Studio > Server Properties > Advanced > Default Language. You can set the default language to Chinese (either Simplified or Traditional) and this will probably fix the problem. However, this is not the easiest option.

You are probably writing the Chinese values into char or varchar columns. Both - char and varchar - support values in server default language encoding (which should be Win-1252 in case of English). Instead of char/varchar you can use nchar/nvarchar that by default store Unicode values - this should solve the problem without affecting all your SQL Server databases/users.

Awesome. I got to learn something today too.

 

 

Reply