Skip to main content
Question

Geojson reader-encoding issue

femo
Contributor
Forum|alt.badge.img+6
  • femoContributor
  • Contributor

Hi,

reading a geojson file from an online ressource delivers the following error:

The file contains german special characters called "Umlaute", such as ä,ö, ü or ß. Manually replacing these characters by other characters, e.g. "ae" instead of ä "solved" the problem. Within FME I can't find any place where one could set the character Encoding.

Any smarter ideas out there?

7 replies

takashi
Contributor
  • takashiContributor
  • August 24, 2017

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

takashi
Contributor
  • takashiContributor
  • August 24, 2017
takashi wrote:

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

Addition: Set "utf-8" to the Target File Encoding parameter in the AttributeFileWriter.

 

This is the overview of the workflow

 

takashi
Contributor
  • takashiContributor
  • August 24, 2017
takashi wrote:

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

Alternatively, this workflow could also work well.

 

Creator -> HTTPCaller -> JSONFragmenter -> GeometryReplacer -> JSONFlattener

 

 

JSONFragmenter Parameters

 

  • JSON Attribute: _response_body
  • JSON Query: json["features"][*]
  • Flatten Query Result into Attributes: Yes
  • Recursively Flatten Objects/Arrays: No
  • Attributes to Expose: geometry properties
GeometryReplacer Parameters

 

  • Geometry Encoding: GeoJSON
  • Geometry Source: @Value(geometry)
JSONFlattener Parameters

 

  • JSON Document: @Value(properties)
  • Attributes to Expose: <expose your desired attributes>
[Addition] I found the "Fragment as Format" parameter in the JSONFragmenter has been restored in FME 2017.1. If you set "GEOJSON" to the parameter, the JSONFragmenter transforms a GeoJSON document directly into features having geometry and attributes. You don't need to use GeometryReplacer and JSONFlattener in FME 2017.1.
danilo_fme
Evangelist
Forum|alt.badge.img+42
  • danilo_fmeEvangelist
  • Evangelist
  • August 24, 2017
takashi wrote:

Hi @femo, unfortunately the current GeoJSON reader doesn't seems to support the encoding. In the interim, this workflow could be a workaround.

  1. Creator: Create an initiator feature.
  2. HTTPCaller: Fetch the GeoJSON document from the URL with GET method and save the response body into an attribute.
  3. AttributeEncoder: Encode the response body with UTF-8.
  4. TempPathnameCreator: Create a temporary file path (extension: geojson).
  5. AttributeFileWriter: Write the response body into the temporary file (Target File Encoding: utf-8).
  6. FeatureReader: Read the temporary file with the GeoJSON reader. Use the Single Output Port <Generic>, and expose desired attribute names through the Attributes to Expose parameter.

Hope this helps.

@takashi, great step-by-step. Congratulations

 

femo
Contributor
Forum|alt.badge.img+6
  • femoContributor
  • Author
  • Contributor
  • August 28, 2017

Hi @takashi, it works perfectly, thanks a lot!

,

deanatsafe
Safer
Forum|alt.badge.img+6
  • deanatsafeSafer
  • Safer
  • January 23, 2019

While Takashi's approach works well, there is another approach that does not require writing / reading external files based on a recent enhancement to JSONFragmenter. Use an HTTPCaller to read this URL, with Response Body Encoding = 'Autodetect from HTTP header'. Then connect the output to a JSONFragmenter, JSONQuery = 'json', fragment as format = 'GEOJSON'. This will allow you to read this data stream directly into FME features, though you will still need to expose attributes that you want visible on the schema.

For more info see:

https://knowledge.safe.com/questions/51479/improve-on-geojson.html

itay
Supporter
Forum|alt.badge.img+17
  • itaySupporter
  • Supporter
  • January 24, 2019
deanatsafe wrote:

While Takashi's approach works well, there is another approach that does not require writing / reading external files based on a recent enhancement to JSONFragmenter. Use an HTTPCaller to read this URL, with Response Body Encoding = 'Autodetect from HTTP header'. Then connect the output to a JSONFragmenter, JSONQuery = 'json', fragment as format = 'GEOJSON'. This will allow you to read this data stream directly into FME features, though you will still need to expose attributes that you want visible on the schema.

For more info see:

https://knowledge.safe.com/questions/51479/improve-on-geojson.html

Nice! @DeanAtSafe

Reply


Helpful Members This Week

Community Stats

31,000
Posts
117,491
Replies
38,778
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.