Skip to main content
Question

Parsing JSON list of ObjectIds from rest service


I need to download some data that's only available through a Map Service on a public ArcGIS server. I was thinking of downloading the data 1000 features at a time using JSON, but in order to do that I need to get the list of ObjectIds. The list of ObjectIds gets returned in a query request and the response looks similar to this:

{ "objectIdFieldName": "OBJECTID", "objectIds": [ 1, 2, 3 ] }

I could parse this using Python, but I would like to be able to do it in FME 2014. When I use the JSON reader, each objectId gets read as a separate attribute and FME ends up hanging because it's trying to read tens of thousands of attributes! Is there a parameter setting or a better reader or transformer that would help?

5 replies

bruceharold
Contributor
Forum|alt.badge.img+17
  • Contributor
  • August 17, 2016

Do you mean a feature service? There is a reader for that, it will handle the pagination for you.


  • Author
  • August 17, 2016
bruceharold wrote:

Do you mean a feature service? There is a reader for that, it will handle the pagination for you.

@bruceharold Unfortunately I'm using FME 2014 and there isn't a reader for feature services on ArcGIS Server.

 

 


itay
Supporter
Forum|alt.badge.img+17
  • Supporter
  • August 17, 2016

How about reading it with the text reader and parsing it with a json transformer?


david_r
Evangelist
  • August 18, 2016
tbuckler wrote:
@bruceharold Unfortunately I'm using FME 2014 and there isn't a reader for feature services on ArcGIS Server.

 

 

Unless you have some very hard constraints, I suspect it will be much less labor-intensive to upgrade FME than to try and re-create the reader. Judging by the number of bugs (I'm using the term lightly) in the early versions of the FME reader I can only assume that accessing the feature service isn't trivial.

 


todd_davis
Supporter
Forum|alt.badge.img+22
  • Supporter
  • August 18, 2016
itay wrote:

How about reading it with the text reader and parsing it with a json transformer?

Yes, read as txt and pass to JsonFlattener...expose attribute: objectIds{}. Next put into a listexploder...

 

 

If you are going down this path, realise that you won't be able to pass heaps of objectids in a querystring (if using GET request) as the default in IIS allows for 2048 bytes....I have some details around the breaking that down into manageable chucks.

 

 

I have a workbench that goes beyond the inbuilt feature service reader capability which I don't mind given out, but it is setup under 2016 now (and avoids the GET Request).

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings