• EN
Question

Parsing JSON list of ObjectIds from rest service

  • 17 August 2016
  • 5 replies
  • 7 views
T

I need to download some data that's only available through a Map Service on a public ArcGIS server. I was thinking of downloading the data 1000 features at a time using JSON, but in order to do that I need to get the list of ObjectIds. The list of ObjectIds gets returned in a query request and the response looks similar to this:

{ "objectIdFieldName": "OBJECTID", "objectIds": [ 1, 2, 3 ] }

I could parse this using Python, but I would like to be able to do it in FME 2014. When I use the JSON reader, each objectId gets read as a separate attribute and FME ends up hanging because it's trying to read tens of thousands of attributes! Is there a parameter setting or a better reader or transformer that would help?

5 replies

bruceharold
Rank
Badge +16

Do you mean a feature service? There is a reader for that, it will handle the pagination for you.

T

Do you mean a feature service? There is a reader for that, it will handle the pagination for you.

@bruceharold Unfortunately I'm using FME 2014 and there isn't a reader for feature services on ArcGIS Server.

 

 

itay
Rank
Badge +16

How about reading it with the text reader and parsing it with a json transformer?

david_r
Rank
Userlevel 5
@bruceharold Unfortunately I'm using FME 2014 and there isn't a reader for feature services on ArcGIS Server.

 

 

Unless you have some very hard constraints, I suspect it will be much less labor-intensive to upgrade FME than to try and re-create the reader. Judging by the number of bugs (I'm using the term lightly) in the early versions of the FME reader I can only assume that accessing the feature service isn't trivial.

 

todd_davis
Rank
Userlevel 2
Badge +12

How about reading it with the text reader and parsing it with a json transformer?

Yes, read as txt and pass to JsonFlattener...expose attribute: objectIds{}. Next put into a listexploder...

 

 

If you are going down this path, realise that you won't be able to pass heaps of objectids in a querystring (if using GET request) as the default in IIS allows for 2048 bytes....I have some details around the breaking that down into manageable chucks.

 

 

I have a workbench that goes beyond the inbuilt feature service reader capability which I don't mind given out, but it is setup under 2016 now (and avoids the GET Request).

 

Reply


Community Stats

28,868
Posts
109,990
Replies
36,680
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.