Unfortunately, the source data can't be obtained as one package. Instead it can only be retrieved through individual SOAP calls. As such, the workflow obtains a dynamically generated list of relevant ID numbers retrieved through an initial GET request and then uses those ID numbers to construct the respective URLs for the individual GET requests, which obtain the data that I need (but it stores the XML retrieved through each GET request as a separate record/attribute). The workflow is for a regular batch update process using the externally sourced data to update my data.
Solved
How do I extract XML that is stored in the _response_body attribute as separate records and use them to make a single table where particular XML tags represent the table fields? I've tried many transformers/approaches with no success...
- Contributor
Best answer by nielsgerrits
gazman wrote:
Hmmmm, yeah - I thought so too. However, I couldn't seem to get either the XMLFragmenter or XMLFlattener to pull out the XML from the _response_body attribute. I'm thinking that I probably haven't referenced the tags correctly to pull out the XML. I haven't used these particular tools before, but my different referencing attempts either didn't run or just created an additional field containing either the same _response_body attribute or empty values.
This is an example of the XML stored against each record. I've had to redact some of the result information due to privacy requirements that exist in relation to the XML content.
The trick with the XMLFragmenter (and XMLFlattener) is to save the response to a sample file, then configure the transformer using the sample file as source, and when configured switch the source to attribute. When you use the file as source, you can select what you want from a tree structure.
See attached workspace template.
Like @caracadrian suggests, saving the response as a text (FeatureWriter, Text File) and reading the written file as XML (FeatureReader) is also an option. But it will be slower due to the extra I/O. If you don't like to clean up the written files you can also use a TempPathnameCreator to write the file to the temp files.
+48- Celebrity
The XMLFragmenter and/or XMLFlattener should be able to do this. Can you post a sample of the XML?
- Contributor
redgeographics wrote:
The XMLFragmenter and/or XMLFlattener should be able to do this. Can you post a sample of the XML?
Hmmmm, yeah - I thought so too. However, I couldn't seem to get either the XMLFragmenter or XMLFlattener to pull out the XML from the _response_body attribute. I'm thinking that I probably haven't referenced the tags correctly to pull out the XML. I haven't used these particular tools before, but my different referencing attempts either didn't run or just created an additional field containing either the same _response_body attribute or empty values.
This is an example of the XML stored against each record. I've had to redact some of the result information due to privacy requirements that exist in relation to the XML content.
+23- Contributor
You can also write the HTTP Response Body as a file and read it via FeatureReader. It's more disk intensive but easier to process than XML Fragmenter/Flattener
- Best Answer
gazman wrote:
Hmmmm, yeah - I thought so too. However, I couldn't seem to get either the XMLFragmenter or XMLFlattener to pull out the XML from the _response_body attribute. I'm thinking that I probably haven't referenced the tags correctly to pull out the XML. I haven't used these particular tools before, but my different referencing attempts either didn't run or just created an additional field containing either the same _response_body attribute or empty values.
This is an example of the XML stored against each record. I've had to redact some of the result information due to privacy requirements that exist in relation to the XML content.
The trick with the XMLFragmenter (and XMLFlattener) is to save the response to a sample file, then configure the transformer using the sample file as source, and when configured switch the source to attribute. When you use the file as source, you can select what you want from a tree structure.
See attached workspace template.
Like @caracadrian suggests, saving the response as a text (FeatureWriter, Text File) and reading the written file as XML (FeatureReader) is also an option. But it will be slower due to the extra I/O. If you don't like to clean up the written files you can also use a TempPathnameCreator to write the file to the temp files.
- Contributor
gazman wrote:
Hmmmm, yeah - I thought so too. However, I couldn't seem to get either the XMLFragmenter or XMLFlattener to pull out the XML from the _response_body attribute. I'm thinking that I probably haven't referenced the tags correctly to pull out the XML. I haven't used these particular tools before, but my different referencing attempts either didn't run or just created an additional field containing either the same _response_body attribute or empty values.
This is an example of the XML stored against each record. I've had to redact some of the result information due to privacy requirements that exist in relation to the XML content.
Thanks so much nielsgerrits! Your explanation and screen prints were exactly what I needed. I'd seen references in another post to saving the response as text, but didn't understand that it was a temporary hack to expose the element tree.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.