Question

encoding

  • 22 July 2019
  • 4 replies
  • 37 views
samluo
Badge

Hi I have a question about writing French Characters (such as é).

Here are my steps (an example)

1. read (FeatureReader) xml file from an API (https://www.arcgis.com/sharing/rest/content/items/05e9d3d5adb64d4cb1447e0ed63e6a43/info/metadata/metadata.xml)

2. here is the mxl file (not a whole file, just from begun to the line with French Character (É)) (see the last 2 lines)

<?xml version="1.0" encoding="UTF-16"?><metadata>

<Esri>

<CreaDate>20181024</CreaDate>

<CreaTime>10205300</CreaTime>

<ArcGISFormat>1.0</ArcGISFormat>

</Esri>

<mdFileID>08d57ccf-ab9c-4006-948c-a458503a057b</mdFileID>

<mdLang>

<languageCode value="fre"></languageCode>

</mdLang>

<mdChar>

<CharSetCd value="004"></CharSetCd>

</mdChar>

<mdHrLv>

<ScopeCd value="005"></ScopeCd>

</mdHrLv>

<mdContact>

<rpIndName>Firstname Lastname</rpIndName>

<rpOrgName>Ministry of Natural Resources</rpOrgName>

<rpPosName>Spatial Data Acquisition Specialist</rpPosName>

<rpCntInfo>

<cntPhone>

<voiceNum>(XXX) XXX-XXXX</voiceNum>

<faxNum></faxNum>

</cntPhone>

<cntAddress>

<delPoint>XXX Water Street, X floor, North Tower</delPoint>

<city>City</city>

<adminArea>Province</adminArea>

<postCode>A1B 2C3</postCode>

<country>CA</country>

<eMailAdd>firstname.lastname@ontario.ca</eMailAdd>

</cntAddress>

<cntInstr></cntInstr>

</rpCntInfo>

<role>

<RoleCd value="007"></RoleCd>

</role>

</mdContact>

<mdDateSt></mdDateSt>

<mdStanName>ArcGIS Metadata</mdStanName>

<mdStanVer>1.0</mdStanVer>

<refSysInfo>

<RefSystem>

<refSysID>

<identCode code="Grid CS = Geographic (Lat, Long); Map Proj = Not Applicable; Horiz. Datum = NAD83; Vert. Datum = Not Applicable"></identCode>

</refSysID>

</RefSystem>

</refSysInfo>

<dataIdInfo>

<idCitation>

<resTitle>Écozones</resTitle>

<resAltTitle>ECOZONE</resAltTitle>

3. Encoding (AttributeEncoder) . I have tried different Destination Encoding but seems making no difference.

4. write (AttributeEncode) to a text file on local computer (file name :ECOZONE_ESRI_FR.xml). when I open the file with NotePad++ , the last 2 lines mentioned above looks totally different (see screenshot)

Can someone please point out what I did wrong or What should I do.

Thanks

Sam

4 replies

samluo
Badge

Here is the screenshot

virtualcitymatt
Rank
Userlevel 4
Badge +26

It looks like your data is in UTF-16. When FME reads it it should pick this up and read the data correctly. Issues like this typically come in the writer. I think by default the writer will use your computers system encoding which may recognise this particular character. To fix the issue you should be able to set the encoding in the writer.

When adding the writer - check the parameters to see if there is an option to set it. If there is set it to be UTF-8 or UTF-16. Both should work fine. If you still find your getting bad output check that FME is reading the file correctly by looking at the data in the FME Data Inspector. If the problem is with reading let us know which format you are reading with.

samluo
Badge

Thanks for comment. Yes there was no problem reading data, from inspector, it shows all the correct characters. but after writing to a file, some French characters are becoming unreadable. I have tried different encoding on Writer (system default, utf-16 and utf-8 and some other not commonly used), but they are all coming with similar results. (each encoding may has some slight difference, I have them all documented all ).

virtualcitymatt
Rank
Userlevel 4
Badge +26

Thanks for comment. Yes there was no problem reading data, from inspector, it shows all the correct characters. but after writing to a file, some French characters are becoming unreadable. I have tried different encoding on Writer (system default, utf-16 and utf-8 and some other not commonly used), but they are all coming with similar results. (each encoding may has some slight difference, I have them all documented all ).

Strange that. FME should just take care of this. I am unable to reproduce the error your having.

 

 

Perhaps you can contact Safe Support and share your data and workspace with them (unless you can share it with us here

Reply


Community Stats

28,669
Posts
109,357
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.