Skip to main content
Question

HttpCaller: send POST and then problems writing download stream to zip file

  • May 30, 2016
  • 10 replies
  • 60 views

peteralstorp
Contributor
Forum|alt.badge.img+7

Hello FME-community. 


I'm having trouble downloading data with the HttpCaller to a zip file. 


1. First I upload text/xml below with a file key to say which file I want to download. See image . 

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetOrder xmlns="http://Vagverket.VIS.Lastkajen.LastkajenLogistics">
      <orderKey>@Value(FileKey)</orderKey>
    </GetOrder>
  </soap:Body>
</soap:Envelope>

2. I then receive (according to the data download specification) a download stream in base64Binary, see example below. 

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <GetOrderResponse xmlns="http://Vagverket.VIS.Lastkajen.LastkajenLogistics">
      <GetOrderResult>base64Binary</GetOrderResult>
    </GetOrderResponse>
  </soap:Body>
</soap:Envelope>

3. I use the XMLFlattener to filter out GetOrderResult, BUT I can't seem to get the download stream into a proper functional zip file. I have tried numerous ways (writing to a file directly from the httpcaller, using the datafile-writer and the attributewriter but I just can't make it work). 

GetOrderResult (base64Binary) follows. This data contains, if I'm correct, a zip file containing a shape file TESTNVDB_DKC_Cykelled.shp (the file contains zero objects), a dbf file TESTNVDB_DKC_Cykelled_domainmap.dbf and a text file.

UEsDBBQAAAAIAG+HuUiDBir8bQAAACIBAAAZACQAVEVTVE5WREJfREtDX0N5a2VsbGVkLmRiZgoAIAAAAAAAAQAYAMlbqgqWttEBNsrpEJa20QE2yukQlrbRAWMuYZVkAAJFxhUMGCCcgcHP0dcPynMGEUZoSgKCPH0d/cJATLA6FjT5IB9PF2T9amjywSGOQSGOYcEhDAxuIL4wN5q8TyhUGoe8h2OAa7yPqx8OebcgR794F8eQUF+I+zjR7A/x9PHBJ88rBQBQSwMEFAAAAAgAb4e5SPQgcxb8AAAAhwEAABkAJABURVNUTlZEQl9ES0NfQ3lrZWxsZWQucHJqCgAgAAAAAAABABgAyVuqCpa20QE2yukQlrbRATbK6RCWttEBZY9Bb4MgGIb/C2diAGuVo6nobKYYtOnBGEIssyTGJsi2vz80WbJ2HDg87/O9HzSCn09tD9orEyynVHYVgAXjxQb9JX8DALO0u1Q9yP6gtnljgpeZV0UrMU0QgMcwTnAYBwgSmgQkigkhGOFhgI0oK+YbCqv18m3GO4AoQAO81GXne/Xk+Y5wfIhCQkmEKT2E4T7Kz+zUlbzuQWfVsn5pu2pZaTsq97DAG6lIK9Yx0YNczT5janVmmQCM0Ha2Rf+c+mHdfZde85NenFXztsHcjFoAxNGL0o5q1jJX4/YAX0ApPT4J78oZ93nTkn9Ibs1klqcPV9ppP4g9GX4AUEsDBBQAAAAIAG+HuUhKJdNaHQAAAGQAAAAZACQAVEVTVE5WREJfREtDX0N5a2VsbGVkLnNocAoAIAAAAAAAAQAYACe+rAqWttEBNsrpEJa20QE2yukQlrbRAWNgUOdiwA6MXjAzMPDikGRg+LAfRv8Hg/f/cdEAUEsDBBQAAAAIAG+HuUhKJdNaHQAAAGQAAAAZACQAVEVTVE5WREJfREtDX0N5a2VsbGVkLnNoeAoAIAAAAAAAAQAYACe+rAqWttEBbyzsEJa20QFvLOwQlrbRAWNgUOdiwA6MXjAzMPDikGRg+LAfRv8Hg/f/cdEAUEsDBBQAAAAIAHGHuUjkPO7Z0wAAALoQAAAjACQAVEVTVE5WREJfREtDX0N5a2VsbGVkX2RvbWFpbm1hcC5kYmYKACAAAAAAAAEAGACDUoYNlrbRAW8s7BCWttEBbyzsEJa20QFjLmGV5GBgYGhkYGdiQAfhDAyOJSVFmUmlJakMDM4gIS40Jc75KakwJoj4hyYflphTmopHnlchIMjT19EvTEFBwVBhJIOC1LyUnMy89OKc1JSBdssAAEQyMBpopwwoSK7MTs0pKS3KLM4diQkBkQyMB9opAwpy80sy8/NGaFmAnAxMBtopAwpQKoX85IwRVjwgkoHpQDtlQAFGMhhZxQMiGZgNtFMGFKBl/pGWEBDJwHygnTKgALk00MGWKIZ3qkAkA8uBdsqAgvzsJ3nDMX6JA1IAUEsDBBQAAAAIAHGHuUixR8oV9QAAAKkBAAAmACQAVEVTVE5WREJfREtDX0N5a2VsbGVkX2ZpZWxkYWxpYXNlcy50eHQKACAAAAAAAAEAGADHjYENlrbRAW8s7BCWttEBbyzsEJa20QGVjz1uwzAMhXcBugPHdOjSGwhwghqwXcM/WQM1oh3BslLQsk/kXCEX0MUqIW7RJUO5kHji9/R40GhUIUcEEEbLKY6cFSIv4FFBsJyVVZqL4hiFkvToVwJpF79aZbXtOauyNNmACjsktJMJr0Po8JEmnNWNqBpxrJu4YKTTiwO5TM7frILO3wkmJ8nBl7+Fifx6Hmb3Sbj511m70U9xMz+n30W5P2X7cFP7K8doPV5HdKSnQULM2yvQ8KY4O1SiOCWiaXNIpJtH6Ch8BdfzBUZUsGgzoIP5jxn0fjUGibMmzbJ/sto6fAQg/DGCndPGvKpo8cLZN1BLAQItABQAAAAIAG+HuUiDBir8bQAAACIBAAAZACQAAAAAAAAAIAAAAAAAAABURVNUTlZEQl9ES0NfQ3lrZWxsZWQuZGJmCgAgAAAAAAABABgAyVuqCpa20QE2yukQlrbRATbK6RCWttEBUEsBAi0AFAAAAAgAb4e5SPQgcxb8AAAAhwEAABkAJAAAAAAAAAAgAAAAyAAAAFRFU1ROVkRCX0RLQ19DeWtlbGxlZC5wcmoKACAAAAAAAAEAGADJW6oKlrbRATbK6RCWttEBNsrpEJa20QFQSwECLQAUAAAACABvh7lISiXTWh0AAABkAAAAGQAkAAAAAAAAACAAAAAfAgAAVEVTVE5WREJfREtDX0N5a2VsbGVkLnNocAoAIAAAAAAAAQAYACe+rAqWttEBNsrpEJa20QE2yukQlrbRAVBLAQItABQAAAAIAG+HuUhKJdNaHQAAAGQAAAAZACQAAAAAAAAAIAAAAJcCAABURVNUTlZEQl9ES0NfQ3lrZWxsZWQuc2h4CgAgAAAAAAABABgAJ76sCpa20QFvLOwQlrbRAW8s7BCWttEBUEsBAi0AFAAAAAgAcYe5SOQ87tnTAAAAuhAAACMAJAAAAAAAAAAgAAAADwMAAFRFU1ROVkRCX0RLQ19DeWtlbGxlZF9kb21haW5tYXAuZGJmCgAgAAAAAAABABgAg1KGDZa20QFvLOwQlrbRAW8s7BCWttEBUEsBAi0AFAAAAAgAcYe5SLFHyhX1AAAAqQEAACYAJAAAAAAAAAAgAAAARwQAAFRFU1ROVkRCX0RLQ19DeWtlbGxlZF9maWVsZGFsaWFzZXMudHh0CgAgAAAAAAABABgAx42BDZa20QFvLOwQlrbRAW8s7BCWttEBUEsFBgAAAAAGAAYAmQIAAKQFAAAAAA==

Any suggestions helping me out here are welcome. 

Kind regards, 
Peter

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

10 replies

takashi
Celebrity
  • May 31, 2016

Hi @peteralstorp, did you try saving the response into a file by the HTTPCaller with setting File to the 'Save Response Body To' parameter?


peteralstorp
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • May 31, 2016

Hi @peteralstorp, did you try saving the response into a file by the HTTPCaller with setting File to the 'Save Response Body To' parameter?

Thanks, takashi. Yes I did. I figure it must be a character encoding problem or maybe I need to understand the zip format better.


takashi
Celebrity
  • May 31, 2016

Thanks, takashi. Yes I did. I figure it must be a character encoding problem or maybe I need to understand the zip format better.

I think you can read the zip file with a FeatureReader following to the HTTPCaller, once you have saved it into the disk. Have you tried this?


peteralstorp
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • May 31, 2016

I think you can read the zip file with a FeatureReader following to the HTTPCaller, once you have saved it into the disk. Have you tried this?

I think the zip file is the problem. I tried your suggestion (although I need to pass an xml-transformer to get rid of some xml-code. See image. takashi.png. The featurereader will not accept the zipfile.


takashi
Celebrity
  • May 31, 2016

I think the zip file is the problem. I tried your suggestion (although I need to pass an xml-transformer to get rid of some xml-code. See image. takashi.png. The featurereader will not accept the zipfile.

How about setting fme-binary to the 'Target File Character Encoding' parameter of the AttributeFileWriter?


peteralstorp
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • May 31, 2016

I think the zip file is the problem. I tried your suggestion (although I need to pass an xml-transformer to get rid of some xml-code. See image. takashi.png. The featurereader will not accept the zipfile.

Tried it but no. The zip file can't be read properly. Error msg: Error status STK_FAILURE pending -- message is: Error while processing folders


takashi
Celebrity
  • May 31, 2016

I think the zip file is the problem. I tried your suggestion (although I need to pass an xml-transformer to get rid of some xml-code. See image. takashi.png. The featurereader will not accept the zipfile.

Well, the TextDecoder might help you to decode the Base64 string to an fme-binary string before writing.

  • Encoding Type: Base64
  • Character Encoding for Binary Data: Binary (fme-binary)

peteralstorp
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • May 31, 2016

I think you can read the zip file with a FeatureReader following to the HTTPCaller, once you have saved it into the disk. Have you tried this?

Hm, still no readable zip file. Thanks for your efforts so far, Takashi!


takashi
Celebrity
  • May 31, 2016

Thanks, takashi. Yes I did. I figure it must be a character encoding problem or maybe I need to understand the zip format better.

I was able to read your sample Base64 string as a zipped Shepfile dataset. See the attachment: base64-to-fme-binary.fmw (FME 2016.1)


peteralstorp
Contributor
Forum|alt.badge.img+7
  • Author
  • Contributor
  • May 31, 2016

I was able to read your sample Base64 string as a zipped Shepfile dataset. See the attachment: base64-to-fme-binary.fmw (FME 2016.1)

Takashi, you are the king! Thank you kindly. I can't yet see what you did different than me, but there must be something!