Skip to main content
Question

Improve on GeoJSON

takashi
Contributor
Forum|alt.badge.img+19

From this Q&A;: Geojson reader-encoding issue

The Source URL: http://stadtplan.bonn.de/geojson?Thema=14403&koordsys;=25832/geojson.json

I think there is a room to improve GeoJSON handling.

  • The GeoJSON reader should support various encodings.
  • The GeoJSON reader and the GeometryReplacer (Geometry Encoding: GeoJSON) should set an appropriate coordinate system to the geometry if the "geometry" member object contains a valid "crs" member, like this.
{
   "type" : "FeatureCollection",
   "features" : [
      {
         "type" : "Feature",
         "geometry" : {
            "type" : "Point",
            "crs" : {
               "type" : "name",
               "properties" : {
                  "name" : "EPSG:25832"
               }
            },
            "coordinates" : [ 369127.5691, 5624581.982 ]
         },
         "properties" : {
...

2 replies

deanatsafe
Safer
Forum|alt.badge.img+6
  • deanatsafeSafer
  • Safer
  • August 25, 2017

Hi @takashi,

Thanks for pointing out these issues. We certainly would like to make it as easy as possible to work with JSON data, wherever it may come from. In regards to encodings, normally the standard for JSON is to use unicode - UTF8 or UTF16. The data stream here is system encoded to charset = ISO-8859-15. Still, the HTTP header does have this information and the reader could in theory use this to help interpret the data. One work around is to use an HTTPCaller to read this URL, with Response Body Encoding = 'Autodetect from HTTP header'. Then connect the output to a JSONFragmenter, JSONQuery = 'json', fragment as format = 'GEOJSON'. This will allow you to read this data stream into FME features. I would agree that ideally the reader should be able to make use of this header content type information as well, so I have filed an enhancement request to that effect. See: readnonutfjson-httpcaller20171.fmw

With regards to the coordinate system, it already is standard functionality that the GeoJSON reader to set an appropriate coordinate system if the geometry member contains a valid CRS. The issue here is that the CRS property is a child member of the geometry and not a property of the parent feature, so the GeoJSON reader didnt find it. We have already made a fix to handle both cases properly in the future. You will see this fix soon in an upcoming FME 2018 beta.

Note that the latest version of the GeoJSON specification will only support LL84 CRS. In FME we will continue to support past specifications and default to using the latest specification for writing. Note the writer parameter - Reproject to WGS84 = Yes by default.

See: https://tools.ietf.org/html/rfc7946#section-4

takashi
Contributor
Forum|alt.badge.img+19
  • takashiContributor
  • Author
  • Contributor
  • August 25, 2017
deanatsafe wrote:

Hi @takashi,

Thanks for pointing out these issues. We certainly would like to make it as easy as possible to work with JSON data, wherever it may come from. In regards to encodings, normally the standard for JSON is to use unicode - UTF8 or UTF16. The data stream here is system encoded to charset = ISO-8859-15. Still, the HTTP header does have this information and the reader could in theory use this to help interpret the data. One work around is to use an HTTPCaller to read this URL, with Response Body Encoding = 'Autodetect from HTTP header'. Then connect the output to a JSONFragmenter, JSONQuery = 'json', fragment as format = 'GEOJSON'. This will allow you to read this data stream into FME features. I would agree that ideally the reader should be able to make use of this header content type information as well, so I have filed an enhancement request to that effect. See: readnonutfjson-httpcaller20171.fmw

With regards to the coordinate system, it already is standard functionality that the GeoJSON reader to set an appropriate coordinate system if the geometry member contains a valid CRS. The issue here is that the CRS property is a child member of the geometry and not a property of the parent feature, so the GeoJSON reader didnt find it. We have already made a fix to handle both cases properly in the future. You will see this fix soon in an upcoming FME 2018 beta.

Note that the latest version of the GeoJSON specification will only support LL84 CRS. In FME we will continue to support past specifications and default to using the latest specification for writing. Note the writer parameter - Reproject to WGS84 = Yes by default.

See: https://tools.ietf.org/html/rfc7946#section-4

Hi @DeanAtSafe, thanks for your clarification.

 

Your workaround - HTTPCaller + JSONFragmenter is the same as the second solution I have suggested in the comment to the original question, but I didn't know the "Fragment as Format" parameter in the JSONFragmenter has been restored in FME 2017.1.

 

I remember that the parameter (called "Explode as Format" before) had been once deprecated in FME 2016. I contacted your Support team at that time.

 

 

C114423 - JSONFragmenter: 'Explode as Format' parameter was deprecated?

 

The answer was: "I spoke with our developer and he mentioned that the Explode As Format option was removed as they were wanting to have the JSONFragmenter be more of a true JSON transformer (i.e. have less to do with Formats and more to do with actually JSON manipulation)."

 

I thought the reason was reasonable, but it's also good that the parameter became available again. Anyway, however, the help doc should be updated to describe about the parameters correctly.

Reply


Helpful Members This Week

Community Stats

30,940
Posts
117,251
Replies
38,742
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.