Question

Format txt output from HTTP Caller

  • 1 November 2019
  • 6 replies
  • 25 views
K
Badge

I am using the HTTP Caller to GET information from an API.

It responds well with a list in a txt file, formatting that list is not easy. I'm able to format in Power BI, but Excel never understands it because it doesn't have a 'common' common delimiter. Except it does, I know that there are three headers in that list that I need.

name

uaid

uri

These repeat throughout the text, and apart form the jargon at the start of the txt file, these seem to act like they should. PowerBi understands this format.

I'd like to use this GET Caller and format the output within my workbench into the three columns so I can use the information to either import into a feature class or simply export as a table.

I'm also looking at using the uaid to do another GET request for more information from this API, but you can't do the second request without the uaid from the first request. I think to do that I need the txt file to be formatted within the workbench.

I've attached a small part of this txt file (it usually has 35000 results) and I've altered info as this is a confidential project, but hopefully this helps.

 

I currently use a FeatureReader to read the txt file (that the http caller saved) back into the workbench, but I'm stuck on how to format it, as above.

1 Attachment

6 replies

daveatsafe
Rank
Userlevel 2
Badge +17

Hi @kimburrows1242,

The text being returned from the HTTPCaller is XML, which can manipulated by FME transformers.

Please add an XMLFragmenter after the HTTPCaller to extract the records, with the following settings:

XML Source Type: Attribute with XML Document
XML Attribute: _response_body
Elements to match: asset
Click on the Flatten Options button and check the Enable Flattening box
Attributes to Expose: asset.name asset.uaid asset.uri

0684Q00000ArKY6QAN.png

The output from the XMLFragmenter will be suitable for writing out to a feature class or table.

K
Badge

Hi @kimburrows1242,

The text being returned from the HTTPCaller is XML, which can manipulated by FME transformers.

Please add an XMLFragmenter after the HTTPCaller to extract the records, with the following settings:

XML Source Type: Attribute with XML Document
XML Attribute: _response_body
Elements to match: asset
Click on the Flatten Options button and check the Enable Flattening box
Attributes to Expose: asset.name asset.uaid asset.uri

0684Q00000ArKY6QAN.png

The output from the XMLFragmenter will be suitable for writing out to a feature class or table.

Works perfectly, thank you. 

The output from this I want to input into a new HTTP Caller using the uaid to GET more info. I have a URL from my client to only GET using one uaid. Could you help me understand how to set this httpcaller up to use more than one ID? 

I've got this as my Request URL:

https://assetregister.XXX.org.uk/ws/v1/asset/asset.uaid?authKey=xxxx-xxxx-xxxx-xxxx

Where I have entered 'asset.uaid' is where it says in their help docs that you should enter the single ID. 

Is this where I'd enter it in the Query String Parameters? 

I've attached a screenshot too. 

0684Q00000ArMQZQA3.jpg

daveatsafe
Rank
Userlevel 2
Badge +17

Works perfectly, thank you.

The output from this I want to input into a new HTTP Caller using the uaid to GET more info. I have a URL from my client to only GET using one uaid. Could you help me understand how to set this httpcaller up to use more than one ID?

I've got this as my Request URL:

https://assetregister.XXX.org.uk/ws/v1/asset/asset.uaid?authKey=xxxx-xxxx-xxxx-xxxx

Where I have entered 'asset.uaid' is where it says in their help docs that you should enter the single ID.

Is this where I'd enter it in the Query String Parameters?

I've attached a screenshot too.

Hi @kimburrows1242,

Please replace 'asset.uaid' in the Request URL with '@Value(asset.uaid)'. This will ensure the contents of asset.uaid are put in the URL. You can remove asset.uaid from the Query String Parameters.

daveatsafe
Rank
Userlevel 2
Badge +17

Works perfectly, thank you.

The output from this I want to input into a new HTTP Caller using the uaid to GET more info. I have a URL from my client to only GET using one uaid. Could you help me understand how to set this httpcaller up to use more than one ID?

I've got this as my Request URL:

https://assetregister.XXX.org.uk/ws/v1/asset/asset.uaid?authKey=xxxx-xxxx-xxxx-xxxx

Where I have entered 'asset.uaid' is where it says in their help docs that you should enter the single ID.

Is this where I'd enter it in the Query String Parameters?

I've attached a screenshot too.

Like this:

K
Badge

Hi @kimburrows1242,

The text being returned from the HTTPCaller is XML, which can manipulated by FME transformers.

Please add an XMLFragmenter after the HTTPCaller to extract the records, with the following settings:

XML Source Type: Attribute with XML Document
XML Attribute: _response_body
Elements to match: asset
Click on the Flatten Options button and check the Enable Flattening box
Attributes to Expose: asset.name asset.uaid asset.uri

0684Q00000ArKY6QAN.png

The output from the XMLFragmenter will be suitable for writing out to a feature class or table.

Hello, my second output includes more elements I want to match, I keep testing them through this by using the Help menu and trying to decipher the xml, but I can't get the split I want. 

I want everything we discussed before using the 'asset' in the 'Elements to match', but I'd also like everything in this xml that follows 'assetLocation' or just these: 

asset.name asset.uaid asset.uri class startChainage endChainage offset osgbEasting osgbNorthing group

newallassets.txt

 

daveatsafe
Rank
Userlevel 2
Badge +17

Hello, my second output includes more elements I want to match, I keep testing them through this by using the Help menu and trying to decipher the xml, but I can't get the split I want.

I want everything we discussed before using the 'asset' in the 'Elements to match', but I'd also like everything in this xml that follows 'assetLocation' or just these:

asset.name asset.uaid asset.uri class startChainage endChainage offset osgbEasting osgbNorthing group

newallassets.txt

 

Another XMLFragmenter, with Elements to Match set to 'asset' and Flattening enabled, should get those attributes for you. You will need to add the following attribute to the Expose Attributes section of the transformer:

  • asset.name
  • asset.uaid
  • asset.uri
  • assetLocation.class
  • assetLocation.startChainage
  • assetLocation.endChainage
  • assetLocation.offset
  • assetLocation.osgbEasting
  • assetLocation.osgbNorthing

There two groups associated with the record, so these are interpreted as a list:

  • assetLocation.attribute{}.group

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.