On my fme server is a workspace for download which I can start with a webhook. I want to use that webhook in a Python Script. Are there any examples for calling the webhook and receiving the data in python?
Best regards Gerhard
Solved
Using Webhook in Python Script?
Best answer by markw
Hi @gkvoelkl! This is certainly possible, but the workflow will depend on the type of data you're outputting from the webhook workspace. For example, are you planning to receive JSON or XML in Python? Here are my findings with different FME Server services:
Data Streaming Service
- In Python, you'll want to use a module like "requests" to make the request to your FME Server webhook URL. More details on W3schools.
- If your webhook workspace outputs a data format like JSON or XML, you can register the Data Streaming service on this workspace so that the response is the output data.
- In your Python, use requests to make a GET call to the webhook and store the response in a variable. You can then set an attribute value to the response body. For example, I added this code below in a PythonCreator in the input method and the "response" attribute needs to be exposed.
webhook_call = requests.get('https://fme-server.com/fmedatastreaming/repository/workspace.fmw?PARAMETER=value&token=tokenvalue')
newFeature = fmeobjects.FMEFeature()
newFeature.setAttribute("response",webhook_call.text)Data Download Service
- I haven't tested out a scenario where you're running a Data Download service workspace, but it would be fairly similar to the approach above. The response body will be HTML that includes a download link to the output ZIP file. You can either use Beautiful Soup in Python to scrape out the URL, or use our transformers like the HTMLExtractor.
- Depending on the data, you may be able to just pass the URL value into a FeatureReader since it can read zipped datasets from URLs.
Please let me know if you have any further questions about this!
Hi @gkvoelkl! This is certainly possible, but the workflow will depend on the type of data you're outputting from the webhook workspace. For example, are you planning to receive JSON or XML in Python? Here are my findings with different FME Server services:
Data Streaming Service
- In Python, you'll want to use a module like "requests" to make the request to your FME Server webhook URL. More details on W3schools.
- If your webhook workspace outputs a data format like JSON or XML, you can register the Data Streaming service on this workspace so that the response is the output data.
- In your Python, use requests to make a GET call to the webhook and store the response in a variable. You can then set an attribute value to the response body. For example, I added this code below in a PythonCreator in the input method and the "response" attribute needs to be exposed.
webhook_call = requests.get('https://fme-server.com/fmedatastreaming/repository/workspace.fmw?PARAMETER=value&token=tokenvalue')
newFeature = fmeobjects.FMEFeature()
newFeature.setAttribute("response",webhook_call.text)Data Download Service
- I haven't tested out a scenario where you're running a Data Download service workspace, but it would be fairly similar to the approach above. The response body will be HTML that includes a download link to the output ZIP file. You can either use Beautiful Soup in Python to scrape out the URL, or use our transformers like the HTMLExtractor.
- Depending on the data, you may be able to just pass the URL value into a FeatureReader since it can read zipped datasets from URLs.
Please let me know if you have any further questions about this!
Hi @Mark Warren
thanks.
I am using Data Download Service. I send the request in Python and get a response.
But there is no link in the response. When I do the web hook in a browser, there is also no link.
Best regards Gerhard
gkvoelkl wrote:
Hi @Mark Warren
thanks.
I am using Data Download Service. I send the request in Python and get a response.
But there is no link in the response. When I do the web hook in a browser, there is also no link.
Best regards Gerhard
Hi @gkvoelkl! The Data Download result with no link means the workspace ran successfully but there were no output features received at the writer(s). I'd suggest taking a look through the job log to see why no features are being received and to compare this with running the workspace in FME Desktop (with the same parameters).
Hi @Mark Warren
now it works.
It was the geometry parameter.
There is no result
That works
Best regards Gerhard
gkvoelkl wrote:
Hi @Mark Warren
now it works.
It was the geometry parameter.
There is no result
That works
Best regards Gerhard
Hi @gkvoelkl, just to clarify the second URL formatting was required to work when running the webhook in Python? If so, do you have a code snippet you can share of how this is being called? And are you using an FME PythonCaller/PythonCreato or a separate script/application? I imagine the webhook URL is supposed to work without needing to reformat the symbol characters in the URL for the geometry parameter.
Hi @Mark Warren ,
I tested the queries in the browser and it was the same result.
The first query wrote into the job log file
2022-7-7 15:27:23 | A JSON syntax error was found at line 1, column 2
2022-7-7 15:27:23 | Unexpected character: 't'. One of the following characters was expected: '"', '}'
2022-7-7 15:27:23 | The value of the import attribute '__GeometryReplacerTemp__' contained invalid JSON, and could not be parsed
Perhaps I have to change something in my workspace.
I am using a separate Python script. I want to integrate it into QGIS.
I will change the geometry parameter into four float parameter (xmin, xmax,ymin,ymax).
That will be easier.
Best Regards Gerhard
gkvoelkl wrote:
Hi @Mark Warren ,
I tested the queries in the browser and it was the same result.
The first query wrote into the job log file
2022-7-7 15:27:23 | A JSON syntax error was found at line 1, column 2
2022-7-7 15:27:23 | Unexpected character: 't'. One of the following characters was expected: '"', '}'
2022-7-7 15:27:23 | The value of the import attribute '__GeometryReplacerTemp__' contained invalid JSON, and could not be parsed
Perhaps I have to change something in my workspace.
I am using a separate Python script. I want to integrate it into QGIS.
I will change the geometry parameter into four float parameter (xmin, xmax,ymin,ymax).
That will be easier.
Best Regards Gerhard
I'd suggest verifying that the GeoJSON is correct if you still want to try this approach. In your previous example, I noticed both URLs have invalid JSON (with a missing comma between the 3rd and 4th set of coordinates). Was this GeoJSON polygon generated outside of FME?
Hi @Mark Warren ,
thank you very much for your help.
with four parameters and without json it is much easier.
Best Regards Gerhard
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.