Skip to main content
Question

encoding


samluo
Forum|alt.badge.img

Hi I have a question about writing French Characters (such as é).

Here are my steps (an example)

1. read (FeatureReader) xml file from an API (https://www.arcgis.com/sharing/rest/content/items/05e9d3d5adb64d4cb1447e0ed63e6a43/info/metadata/metadata.xml)

2. here is the mxl file (not a whole file, just from begun to the line with French Character (É)) (see the last 2 lines)

<?xml version="1.0" encoding="UTF-16"?><metadata>

<Esri>

<CreaDate>20181024</CreaDate>

<CreaTime>10205300</CreaTime>

<ArcGISFormat>1.0</ArcGISFormat>

</Esri>

<mdFileID>08d57ccf-ab9c-4006-948c-a458503a057b</mdFileID>

<mdLang>

<languageCode value="fre"></languageCode>

</mdLang>

<mdChar>

<CharSetCd value="004"></CharSetCd>

</mdChar>

<mdHrLv>

<ScopeCd value="005"></ScopeCd>

</mdHrLv>

<mdContact>

<rpIndName>Firstname Lastname</rpIndName>

<rpOrgName>Ministry of Natural Resources</rpOrgName>

<rpPosName>Spatial Data Acquisition Specialist</rpPosName>

<rpCntInfo>

<cntPhone>

<voiceNum>(XXX) XXX-XXXX</voiceNum>

<faxNum></faxNum>

</cntPhone>

<cntAddress>

<delPoint>XXX Water Street, X floor, North Tower</delPoint>

<city>City</city>

<adminArea>Province</adminArea>

<postCode>A1B 2C3</postCode>

<country>CA</country>

<eMailAdd>firstname.lastname@ontario.ca</eMailAdd>

</cntAddress>

<cntInstr></cntInstr>

</rpCntInfo>

<role>

<RoleCd value="007"></RoleCd>

</role>

</mdContact>

<mdDateSt></mdDateSt>

<mdStanName>ArcGIS Metadata</mdStanName>

<mdStanVer>1.0</mdStanVer>

<refSysInfo>

<RefSystem>

<refSysID>

<identCode code="Grid CS = Geographic (Lat, Long); Map Proj = Not Applicable; Horiz. Datum = NAD83; Vert. Datum = Not Applicable"></identCode>

</refSysID>

</RefSystem>

</refSysInfo>

<dataIdInfo>

<idCitation>

<resTitle>Écozones</resTitle>

<resAltTitle>ECOZONE</resAltTitle>

3. Encoding (AttributeEncoder) . I have tried different Destination Encoding but seems making no difference.

4. write (AttributeEncode) to a text file on local computer (file name :ECOZONE_ESRI_FR.xml). when I open the file with NotePad++ , the last 2 lines mentioned above looks totally different (see screenshot)

Can someone please point out what I did wrong or What should I do.

Thanks

Sam

4 replies

samluo
Forum|alt.badge.img
  • Author
  • July 22, 2019

Here is the screenshot


virtualcitymatt
Celebrity
Forum|alt.badge.img+35

It looks like your data is in UTF-16. When FME reads it it should pick this up and read the data correctly. Issues like this typically come in the writer. I think by default the writer will use your computers system encoding which may recognise this particular character. To fix the issue you should be able to set the encoding in the writer.

When adding the writer - check the parameters to see if there is an option to set it. If there is set it to be UTF-8 or UTF-16. Both should work fine. If you still find your getting bad output check that FME is reading the file correctly by looking at the data in the FME Data Inspector. If the problem is with reading let us know which format you are reading with.


samluo
Forum|alt.badge.img
  • Author
  • July 30, 2019

Thanks for comment. Yes there was no problem reading data, from inspector, it shows all the correct characters. but after writing to a file, some French characters are becoming unreadable. I have tried different encoding on Writer (system default, utf-16 and utf-8 and some other not commonly used), but they are all coming with similar results. (each encoding may has some slight difference, I have them all documented all ).


virtualcitymatt
Celebrity
Forum|alt.badge.img+35
samluo wrote:

Thanks for comment. Yes there was no problem reading data, from inspector, it shows all the correct characters. but after writing to a file, some French characters are becoming unreadable. I have tried different encoding on Writer (system default, utf-16 and utf-8 and some other not commonly used), but they are all coming with similar results. (each encoding may has some slight difference, I have them all documented all ).

Strange that. FME should just take care of this. I am unable to reproduce the error your having.

 

 

Perhaps you can contact Safe Support and share your data and workspace with them (unless you can share it with us here

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings