Question

Geojson reader-encoding issue

  • 24 August 2017
  • 7 replies
  • 18 views
F
Badge

Hi,

reading a geojson file from an online ressource delivers the following error:

The file contains german special characters called "Umlaute", such as ä,ö, ü or ß. Manually replacing these characters by other characters, e.g. "ae" instead of ä "solved" the problem. Within FME I can't find any place where one could set the character Encoding.

Any smarter ideas out there?

7 replies

takashi
Rank
Userlevel 2
Badge +17

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

takashi
Rank
Userlevel 2
Badge +17

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

Addition: Set "utf-8" to the Target File Encoding parameter in the AttributeFileWriter.

 

This is the overview of the workflow

 

takashi
Rank
Userlevel 2
Badge +17

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

Alternatively, this workflow could also work well.

 

Creator -> HTTPCaller -> JSONFragmenter -> GeometryReplacer -> JSONFlattener

 

 

JSONFragmenter Parameters

 

  • JSON Attribute: _response_body
  • JSON Query: json["features"][*]
  • Flatten Query Result into Attributes: Yes
  • Recursively Flatten Objects/Arrays: No
  • Attributes to Expose: geometry properties
GeometryReplacer Parameters

 

  • Geometry Encoding: GeoJSON
  • Geometry Source: @Value(geometry)
JSONFlattener Parameters

 

  • JSON Document: @Value(properties)
  • Attributes to Expose: <expose your desired attributes>
[Addition] I found the "Fragment as Format" parameter in the JSONFragmenter has been restored in FME 2017.1. If you set "GEOJSON" to the parameter, the JSONFragmenter transforms a GeoJSON document directly into features having geometry and attributes. You don't need to use GeometryReplacer and JSONFlattener in FME 2017.1.
danilo_fme
Rank
Userlevel 4
Badge +30

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

@takashi, great step-by-step. Congratulations

 

F
Badge

Hi @takashi, it works perfectly, thanks a lot!

,

deanatsafe
Rank
Badge +6

While Takashi's approach works well, there is another approach that does not require writing / reading external files based on a recent enhancement to JSONFragmenter. Use an HTTPCaller to read this URL, with Response Body Encoding = 'Autodetect from HTTP header'. Then connect the output to a JSONFragmenter, JSONQuery = 'json', fragment as format = 'GEOJSON'. This will allow you to read this data stream directly into FME features, though you will still need to expose attributes that you want visible on the schema.

For more info see:

https://knowledge.safe.com/questions/51479/improve-on-geojson.html

itay
Rank
Badge +16

While Takashi's approach works well, there is another approach that does not require writing / reading external files based on a recent enhancement to JSONFragmenter. Use an HTTPCaller to read this URL, with Response Body Encoding = 'Autodetect from HTTP header'. Then connect the output to a JSONFragmenter, JSONQuery = 'json', fragment as format = 'GEOJSON'. This will allow you to read this data stream directly into FME features, though you will still need to expose attributes that you want visible on the schema.

For more info see:

https://knowledge.safe.com/questions/51479/improve-on-geojson.html

Nice! @DeanAtSafe

Reply


Community Stats

28,753
Posts
109,618
Replies
36,587
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.