Skip to main content
Solved

HTTP call JSON parsing problem

Z
Forum|alt.badge.img+1

I have a problem with parsing JSON which is the result of an HTTP call to get what I need. (See json files attached.)

If I download the .json file and simply put it in a JSON reader with a query,

filter for type:"Update Feature Attributes" and explode some of the lists, it gives me what I need:

My problem is that I have loads of transactions to go through the same way, so I'd need to use an HTTPCaller. I've managed to get that far that with a JSONExtractor I can get "type", "ID" for example, but struggling to break down lists "attributesAfterChange" and "changedAttributesNames" - ListExploder doesn't recognise them as lists for some reason.

Any ideas what should I do to get the list values/break down the arrays? Maybe it's very simple but I just cannot find a solution right now. NB: the attributes in these lists vary and not exclusive to what's in the attached json files, so somewhat making it a bit more complex...

Any help is very much appreciated, thanks in advance.

Zoltan

Best answer by nathanatsafe

Hi @zkovacs,

 

Depending on how the rest of your workspace is built and how you construct your HTTP calls, one option might be to use the FeatureReader instead of a combination of HTTPCaller and JSON transformers. In the FeatureReader you can set the format to JSON or GeoJSON and use a URL for the source dataset parameter (this could also be an attribute or a string built from attributes using the text editor. This might save you some steps.

 

Hope this helps.

 

Best,

 

Nathan
View original
Did this help you find an answer to your question?

4 replies

H
Forum|alt.badge.img+2
Hi @zkovacs,

 

 

When you set up the list attributes to be exposed in the JSONFragmenter are you including the curly brackets {} ?
nathanatsafe
Safer
Forum|alt.badge.img+7
  • nathanatsafeSafer
  • Safer
  • Best Answer
  • August 23, 2018

Hi @zkovacs,

 

Depending on how the rest of your workspace is built and how you construct your HTTP calls, one option might be to use the FeatureReader instead of a combination of HTTPCaller and JSON transformers. In the FeatureReader you can set the format to JSON or GeoJSON and use a URL for the source dataset parameter (this could also be an attribute or a string built from attributes using the text editor. This might save you some steps.

 

Hope this helps.

 

Best,

 

Nathan
Z
Forum|alt.badge.img+1
  • zkovacs
  • Author
  • August 28, 2018
nathanatsafe wrote:

Hi @zkovacs,

 

Depending on how the rest of your workspace is built and how you construct your HTTP calls, one option might be to use the FeatureReader instead of a combination of HTTPCaller and JSON transformers. In the FeatureReader you can set the format to JSON or GeoJSON and use a URL for the source dataset parameter (this could also be an attribute or a string built from attributes using the text editor. This might save you some steps.

 

Hope this helps.

 

Best,

 

Nathan
Hi Nathan,

 

 

Thanks for the tip. Unfortunately I cannot use a FeatureReader as I need to know if a request to the API I'm calling was successful (HTTP 200) or not, so I can set up a loop to keep trying until I get all the transactions I need.

 

I set up an HTTP Caller with a loop to do this and it generally works OK (apart from the minor issue of trying to break up the mentioned arrays/lists) but for some reason, some of the features are 'objects' instead of 'json_line' and I cannot figure out why...

 

I've attached the fmw file and the csv containing the transaction IDs, could you have a look and see if you can find out why some of the features are objects? I had a look at the response and I couldn't see anything in particular that would be the cause.

 

 

Thanks,

 

Zoltan

 

 

[attachments removed]
Z
Forum|alt.badge.img+1
  • zkovacs
  • Author
  • August 31, 2018

I've managed to work around the issues I had by downloading the .json files from the API using a Python script I wrote and then did the rest of the workflow in FME.

Reply


Helpful Members This Week

Community Stats

30,955
Posts
117,307
Replies
38,743
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.