Skip to main content
  • EN
  • Home
  • Forums
  • FME Form
  • General
  • Is it possible to pass in Published Parameters to the Data Download Service with an HTTP POST request?

I would like to know how to build the JSON body for an HTTP POST request to pass in Published Workspace Parameters, opt_requesteremail (who to send an email to), and nm_jobsuccess_topic (which topic to notify on success) for the Data Download Service. 

 

The goal is to interface with another web application where a user can  to submit an asynchronous "report request," that emails a download link when their report has finished processing.

 

I am specifically looking at the POST to avoid any potential issues with large/complex geometries being passed in as a parameter and being too long for the URL.

 

I had started looking into the FME REST API, and figured out how to define the parameters in a JSON body, but eventually ran into an article that states "A Web Service URL looks similar to the REST API Request URL; however, it exists outside of the REST API. A Web Service URL should be used instead of a REST call if you are intending on using the Data Streaming or Data Download Service. A job submitted by a REST call will not have access to these services." 

 

That JSON body for my REST call looked like .

{
      "publishedParameters":  
        {
          "name": "EMAIL",
          "value": "$(EMAIL)"
        },
        {
            "name": "QUERY_AREA",
            "value": '$(AREA)'
        },
        {
            "name": "FEATURES_TO_EXPORT",
            "value": Â"$(FEATURES_TO_EXPORT)"]
        }
      ],
  "NMDirectives": {
    "directives": "
      {
        "name": "email_to",
        "value": "$(EMAIL)"
      }
    ],
    "successTopics": Â
      "Report_Download_Service_Topic"
    ],
    "failureTopics": :]
  }
}

How would I pass the same information to the Data Download Service?

 

Thank you for your time!

Michael Summerton

It's super annoying that the DataDownload Service doesn't support this kind of request. We have the same kind of problems

 

You can configure a Reader to consume the body of a POST request in thew DataDownload service - This is how we get around it (I don't like it):

 

For strings/parameters we expect to be long we put these in the POST body - this post body gets fed to a JSON Reader in the workspace - it's definitely a drawback not having these as parameters.

 

We still need to send the rest of the parameters (e.g., opt_requesteremail) as query string parameters.

 

the DataDownload service has a few other draw backs, like the fact that you can't configure the name of the zip file etc.

 

A better process would be to use the REST API to properly consume the POST request and handle the creating of the zip (or indeed any other output yourself). This lets you configure the service as you like but at the cost of more dev effort on your side.

 

We chose the much less elegant solution to get around the issue and I'm not wild about it. I'd love it if the Services were able to support POST requests in the same was the REST_API can.

It's super annoying that the DataDownload Service doesn't support this kind of request. We have the same kind of problems

 

You can configure a Reader to consume the body of a POST request in thew DataDownload service - This is how we get around it (I don't like it):

 

For strings/parameters we expect to be long we put these in the POST body - this post body gets fed to a JSON Reader in the workspace - it's definitely a drawback not having these as parameters.

 

We still need to send the rest of the parameters (e.g., opt_requesteremail) as query string parameters.

 

the DataDownload service has a few other draw backs, like the fact that you can't configure the name of the zip file etc.

 

A better process would be to use the REST API to properly consume the POST request and handle the creating of the zip (or indeed any other output yourself). This lets you configure the service as you like but at the cost of more dev effort on your side.

 

We chose the much less elegant solution to get around the issue and I'm not wild about it. I'd love it if the Services were able to support POST requests in the same was the REST_API can.

Ah, thank you @virtualcitymatt​, I really appreciate the thorough reply . That really helps clarify that the HTTP POST must be used differently for the DataDownload Service than the REST Service. The HTTP POST/REST stuff is newer territory for me, and your explanation was able to help fill in some of the gaps. It sounds like I've reached the same fork in our development, to decide on :

  • Using the DataDownload Service with a workaround like the one you describe
    • Any simple input parameters can be passed in via the URL, and will be accessible as "Published Parameters" inside the workbench
    • Any "Service Specific Request Parameters" (like opt_requesteremail) or Notification Directives (like nm_jobsuccess_topic) can also be passed in via the URL
    • Any complex/long input parameters could be built in as a POST, and read in with a Reader. In this case, I can use a JSON Reader since I already have most of the JSON built, I'll just need to move my other simple parameters to the URL. These more complex parameters can be read from the "Features" created by the JSON Reader, and managed as Feature "Attributes." I definitely agree, and have experienced the drawbacks of using Attributes to carry Parameters. In my specific example, my complex input is the geometry of a polygon (whichever was selected by the user) which I wouldn't have any issues using it via the Reader (I didn't understand how the POST to the Reader worked at all before your explanation).
      • Can formulate the JSON/POST any way that an FME Reader could read. I should think of the POST Body as the "contents of a file" and the Reader Type as the "format of that file"
  • Using the REST API and putting in the development effort to do our own file delivery
    • Developing a "Report Portal" into our web application has been on the table as a dream solution, but we thought we'd try to take advantage of the DataDownload service that already exists. Now understanding the difference in the HTTP POST for the REST vs DataDownload helps understand what's currently capable and what would need to be developed.

 

I will consider this information and take it back to the team to decide which direction we want to head next. I can see us starting out with what the DataDownload Service already provides, with the potential of developing our own "Report Portal" for users to reach their reports.

 

Thanks again! The explanation of how you worked around the problem was very helpful.

Michael

Perhaps @rylanatsafe​ might be able to point us in a new direction or give us some pointers in case we're not thinking about this in the right way or if there another method?

​

​

Perhaps @rylanatsafe​ might be able to point us in a new direction or give us some pointers in case we're not thinking about this in the right way or if there another method?

​

​

@virtualcitymatt​  I think it's rather clever what you have managed to do, given the FME Flow Data Download Web Service is not able to parse messages (at least how it would normally be expected with a RESTful API). Direct interactions with Flow Web Services (whether it be fmejobsubmitter, fmedatadownload, fmedatastreaming) had not been designed with broader application integration scenarios in mind—rather, I believe that they were created to support interaction within the Web Interface.

Thank you for tagging me into this discussion thread! We currently don't have anything on the roadmap that will make this easier to accomplish.

Reply


Community Stats

30,162
Posts
114,482
Replies
37,809
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.