Presumably tha scripting language is Python?

If so then you will need to create a webhook in FME server and get an API token for the job.

​You will then need to use python to build the request Url from the user parameters including the API token.

Once you have the URL sorted then you can fire off the request. The response will include the URL for downloading the zip.

You will then need to use python to download it and extract it.

​This is just the summary of steps. Writing the script will be the real task but this should at least give you a framework to start from.

Hope it helps​

No sorry, this doesn't really help. Id like'to have examples and experiences from people who have actually done this. The steps I could figure out myself. I even made a script that actually works. It calls a fmedatadownload service with some parameters and it returns with a download URL. I managed to download the data and show it in QGIS.

But I have a few problems/questions:

1. When the workspace is asked for a larger dataset, I get a timeout and no data download URL! How can I prevent this from happening?
2. I use a data download, but maybe its better to use data streaming?
3. I use a requests.post, but maybe it needs a get.
4. How can I upload data, like a shapefile, to FME server from QGIS?
5. There is a Rest API, but I don't see any examples on how to use this with Python.

First off, here is a nice litte github repo with how to use Python and FME Server - I think you should find stuff in here for uploading: https://github.com/bcgov/dbc-pylib/blob/master/docs/FMEServer.md

And yeah, the timeout is a pain:

You might want to consider using: opt_servicemode=async in your request.

The problem here is that you will not get a download link in the response. But, you will get the job id. You can then poll the job table every few seconds until the job is successful, in that case the result will include the download link. Not ideal, but should work.

DataStreaming can still timeout so not the best solution, I don't think. Stick with datadownload if you can't just write directly to the output.

Thank you for yoyr reply,

I've added the pt_servicemode=async in my request and get back:

{"serviceResponse": {

  "jobID": 292954,

  "statusInfo": {

   "mode": "async",

   "status": "success"

  }

}}

Status says it's success but it's not finished yet. How do I go from here. Is there a way to find out when this job is finished and get the download URL?

OK so the "status": "success" in your result above just means that the job was submitted successfully not that it is complete here. Async mode means that the request wont wait for the job to finish.

You will need to use the returned job ID to make periodic requests to the job table using this call (perhaps every 5 seconds):

https://docs.safe.com/fme/html/FME_REST/apidoc/v3/index.html#!/transformations/result_get_22

This will get you the result details including the download url in the case of a successful result.

Hi  ​@arnovananrooij ,

Just to understand what you are doing exactly; I assume the following:

1. You are working with QGIS on Project A, then you need to download some of the data or all from FME Flow.
2. Question; what is your selection criteria from FME Flow? Download all the data?
    

You could stream the data as well, however you need to consider the volume of the data you need to stream to load into QGIS assuming that you are located in Europe and the data is located in Canada.

I am working on something towards this line please give me more info that might help.

Cheers.

Lyes.

Hi ​@mygis,

As per discussion above between virtualcitymatt and arnovananrooij: Using a webhook URL from an FME Flow workspace to return data (like GeoJSON or SHP) to QGIS doesn’t work directly with the “Add Vector Layer > File/HTTPS” option. That’s because QGIS expects a direct URL to a file (e.g., https://server.com/file.geojson), whereas an FME Flow webhook URL simply executes a workspace. Even if you use the Data Streaming or Data Download service, the user still needs to create a Python script that runs the workspace, waits for the result, and then loads the output as a QGIS layer.

Assuming that by “selection criteria”, you’re referring to Published Parameters in the FME Flow workspace that filter the data returned to QGIS via the webhook URL (for example, filtering features within a bounding box or with specific attribute values). As an alternative, you could consider:

• Publishing a WMS or WFS endpoint from FME Flow (as shown in this article), or

• Using Data Virtualization in FME to build a lightweight GeoJSON API that QGIS can read directly.

It’s not the most straightforward setup, but that’s generally the level of effort needed to “stream” data into a GIS application using FME Flow as the data source.

As an aside, if you want to explore OGC-based integrations, you can also look into con terra’s OGC API packages on FME Hub. These enable working with OGC API Features and OGC API Processes.

Best regards,

Hi ​@macatsafe . I agree with you, however a third option would be to use python by calling a workspace through the streaming service, sending the data in json/geojson and receiving it the same way. There is some configuration to be done at the workspace level of course.

Thanks.

Regards.

Lyes.