There is messy code when I open a shapefile created by arcgisdesktop, but it shows well in arcgisdesktop. I don't know why. I hope someone can help me to solves this problem , thanks a lot.Here is the testfile.

There is messy code when I open a shapefile created by arcgisdesktop

Hi @liujisheng, it seems that you encountered the known issue that the FME Shapefile Reader/Writer always handles attribute names with system default encoding. See also here.

ESRI Shape files with UTF-8 encoded field names ?

The situation could appear if the Shapefile dataset was created by ArcGIS with UTF-8 and the default encoding of your machine was other one (e.g. GB2312/18030 in the case of Chinese Windows).

Unfortunately there is no way to solve it unless you re-create the dataset with the same encoding as the default encoding of the machine you run FME.

Hope this will be fixed as soon as possible. @daleatsafe, @LenaAtSafe

The encoding of your file is UTF-8. Unfortunately, the current FME can read the attribute value according to UTF-8 according to the cpg file, but by default, the attribute name is read in the default encoding of the OS (In your case, it should be GBK or GB2312).

Please refer to @takashi's answer. This problem has existed for a long time, but unfortunately, it seems that it has still been solved.

https://knowledge.safe.com/questions/55871/esri-shape-files-with-utf-8-encoded-field-names.html

All of them will cause garbled attribute names. To avoid this problem, I think you can try the following method.

1. Use QGIS to convert your Shape to GBK or GB2312 encoded Shape

2. Read the converted Shape with FME

Hi @liujisheng, it seems that you encountered the known issue that the FME Shapefile Reader/Writer always handles attribute names with system default encoding. See also here.

ESRI Shape files with UTF-8 encoded field names ?

The situation could appear if the Shapefile dataset was created by ArcGIS with UTF-8 and the default encoding of your machine was other one (e.g. GB2312/18030 in the case of Chinese Windows).

Unfortunately there is no way to solve it unless you re-create the dataset with the same encoding as the default encoding of the machine you run FME.

Hope this will be fixed as soon as possible. @daleatsafe, @LenaAtSafe

@takashi

Oh(^.^).

I also very much hope that this issue can be resolved as soon as possible.

Hello all, This still remains a problem and sadly is a challenge for us. We're exploring some options but unfortunately you'll have to workaround for FME 2018. If you look at the last comment on this post https://www.quora.com/Why-doesnt-Microsoft-use-UTF-8-on-Windows-10 -- if you are using Windows 10 and you click that box, your problems may go away. Not 100% the solution but one that may work for you in the short term.

Hi @liujisheng, it seems that you encountered the known issue that the FME Shapefile Reader/Writer always handles attribute names with system default encoding. See also here.

ESRI Shape files with UTF-8 encoded field names ?

The situation could appear if the Shapefile dataset was created by ArcGIS with UTF-8 and the default encoding of your machine was other one (e.g. GB2312/18030 in the case of Chinese Windows).

Unfortunately there is no way to solve it unless you re-create the dataset with the same encoding as the default encoding of the machine you run FME.

Hope this will be fixed as soon as possible. @daleatsafe, @LenaAtSafe

Hello all, This still remains a problem and sadly is a challenge for us. We're exploring some options but unfortunately you'll have to workaround for FME 2018. If you look at the last comment on this post https://www.quora.com/Why-doesnt-Microsoft-use-UTF-8-on-Windows-10 -- if you are using Windows 10 and you click that box, your problems may go away. Not 100% the solution but one that may work for you in the short term.

Sadly this is a very very tough one to win. One hopeful future is this latest Windows 10 feature. If you have Windows 10 and this update, please give that a try. In our early testing it appears to resolve issues like this. Long run we have more work to do in this area, but this new Windows 10 option does provide an alternate and robust way for FME to work well in this type of scenario.

It's known that the problem is caused by that FME Shapefile reader/writer attempts to read/write attribute names always with the system default encoding, regardless of "Character Encoding" parameter setting.I reported the issue to Safe support at least two times in the past 2-3 years and relevant PRs should have been filed already.The symptom could appear in several situations. For example, in Japanese case, the dataset containing Japanese characters (Shift JIS = Windows System encoding) within attribute names cannot be read/written correctly with FME on Linux or MacOS whose system encoding is UTF-8. This indicates that FME Server for Linux (including FME Cloud) cannot be used to translate Shapefile datasets that could contain non-ASCII attribute names.

Just hope you understand how serious this issue is.

Reply

Community Stats

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded