Solved

Using Webhook in Python Script?

G
Badge

On my fme server is a workspace for download which I can start with a webhook. I want to use that webhook in a Python Script. Are there any examples for calling the webhook and receiving the data in python?

 

Best regards Gerhard

icon

Best answer by markw 5 July 2022, 20:45

View original

8 replies

markw
Badge

Hi @gkvoelkl​!  This is certainly possible, but the workflow will depend on the type of data you're outputting from the webhook workspace.  For example, are you planning to receive JSON or XML in Python?  Here are my findings with different FME Server services:

 

Data Streaming Service

  • In Python, you'll want to use a module like "requests" to make the request to your FME Server webhook URL.  More details on W3schools.
  • If your webhook workspace outputs a data format like JSON or XML, you can register the Data Streaming service on this workspace so that the response is the output data. 
  • In your Python, use requests to make a GET call to the webhook and store the response in a variable.  You can then set an attribute value to the response body.  For example, I added this code below in a PythonCreator in the input method and the "response" attribute needs to be exposed.
webhook_call = requests.get('https://fme-server.com/fmedatastreaming/repository/workspace.fmw?PARAMETER=value&token=tokenvalue')
newFeature = fmeobjects.FMEFeature()
newFeature.setAttribute("response",webhook_call.text)

Data Download Service

  • I haven't tested out a scenario where you're running a Data Download service workspace, but it would be fairly similar to the approach above.  The response body will be HTML that includes a download link to the output ZIP file.  You can either use Beautiful Soup in Python to scrape out the URL, or use our transformers like the HTMLExtractor.  
  • Depending on the data, you may be able to just pass the URL value into a FeatureReader since it can read zipped datasets from URLs.  

 

Please let me know if you have any further questions about this!

G
Badge

Hi @Mark Warren​ 

thanks.

I am using Data Download Service. I send the request in Python and get a response.

But there is no link in the response. When I do the web hook in a browser, there is also no link.

E8847F4D-B55F-48E9-9671-EDA0751B1F38 

Best regards Gerhard

markw
Badge

Hi @Mark Warren​ 

thanks.

I am using Data Download Service. I send the request in Python and get a response.

But there is no link in the response. When I do the web hook in a browser, there is also no link.

E8847F4D-B55F-48E9-9671-EDA0751B1F38 

Best regards Gerhard

Hi @gkvoelkl! The Data Download result with no link means the workspace ran successfully but there were no output features received at the writer(s). I'd suggest taking a look through the job log to see why no features are being received and to compare this with running the workspace in FME Desktop (with the same parameters).

G
Badge

Hi @Mark Warren​ 

now it works.

 

It was the geometry parameter.

 

There is no result

 

https://...fmw?Geom_Geometrie={"type":"Polygon","coordinates":[[[12.092686,49.015517],[12.092686,49.01743],[12.095947,49.01743][12.095947,49.015517],[12.092686,49.015517]]]}&NUM_HORIZONTAL_TILES=0&NUM_VERTICAL_TILES=0&opt_showresult=false&opt_servicemode=sync&token=..

 

That works

 

https://...fmw?Geom_Geometrie="<opencurly><quote>type<quote>:<quote>Polygon<quote><comma><quote>coordinates<quote>:<openbracket><openbracket><openbracket>12.092686<comma>49.015517<closebracket><comma><openbracket>12.092686<comma>49.01743<closebracket><comma><openbracket>12.095947<comma>49.01743<closebracket><comma><openbracket>12.095947<comma>49.015517<closebracket><comma><openbracket>12.092686<comma>49.015517<closebracket><closebracket><closebracket><closecurly>"&NUM_HORIZONTAL_TILES=0&NUM_VERTICAL_TILES=0&opt_showresult=false&opt_servicemode=sync&token=..

 

Best regards Gerhard

 

markw
Badge

Hi @Mark Warren​ 

now it works.

 

It was the geometry parameter.

 

There is no result

 

https://...fmw?Geom_Geometrie={"type":"Polygon","coordinates":[[[12.092686,49.015517],[12.092686,49.01743],[12.095947,49.01743][12.095947,49.015517],[12.092686,49.015517]]]}&NUM_HORIZONTAL_TILES=0&NUM_VERTICAL_TILES=0&opt_showresult=false&opt_servicemode=sync&token=..

 

That works

 

https://...fmw?Geom_Geometrie="<opencurly><quote>type<quote>:<quote>Polygon<quote><comma><quote>coordinates<quote>:<openbracket><openbracket><openbracket>12.092686<comma>49.015517<closebracket><comma><openbracket>12.092686<comma>49.01743<closebracket><comma><openbracket>12.095947<comma>49.01743<closebracket><comma><openbracket>12.095947<comma>49.015517<closebracket><comma><openbracket>12.092686<comma>49.015517<closebracket><closebracket><closebracket><closecurly>"&NUM_HORIZONTAL_TILES=0&NUM_VERTICAL_TILES=0&opt_showresult=false&opt_servicemode=sync&token=..

 

Best regards Gerhard

 

Hi @gkvoelkl​, just to clarify the second URL formatting was required to work when running the webhook in Python? If so, do you have a code snippet you can share of how this is being called? And are you using an FME PythonCaller/PythonCreato or a separate script/application? I imagine the webhook URL is supposed to work without needing to reformat the symbol characters in the URL for the geometry parameter.

G
Badge

Hi @Mark Warren​ ,

I tested the queries in the browser and it was the same result.

The first query wrote into the job log file

 

2022-7-7 15:27:23 | A JSON syntax error was found at line 1, column 2

2022-7-7 15:27:23 | Unexpected character: 't'. One of the following characters was expected: '"', '}'

2022-7-7 15:27:23 | The value of the import attribute '__GeometryReplacerTemp__' contained invalid JSON, and could not be parsed

 

Perhaps I have to change something in my workspace.

 

I am using a separate Python script. I want to integrate it into QGIS.

I will change the geometry parameter into four float parameter (xmin, xmax,ymin,ymax).

That will be easier.

 

Best Regards Gerhard

markw
Badge

Hi @Mark Warren​ ,

I tested the queries in the browser and it was the same result.

The first query wrote into the job log file

 

2022-7-7 15:27:23 | A JSON syntax error was found at line 1, column 2

2022-7-7 15:27:23 | Unexpected character: 't'. One of the following characters was expected: '"', '}'

2022-7-7 15:27:23 | The value of the import attribute '__GeometryReplacerTemp__' contained invalid JSON, and could not be parsed

 

Perhaps I have to change something in my workspace.

 

I am using a separate Python script. I want to integrate it into QGIS.

I will change the geometry parameter into four float parameter (xmin, xmax,ymin,ymax).

That will be easier.

 

Best Regards Gerhard

I'd suggest verifying that the GeoJSON is correct if you still want to try this approach. In your previous example, I noticed both URLs have invalid JSON (with a missing comma between the 3rd and 4th set of coordinates). Was this GeoJSON polygon generated outside of FME?

G
Badge

Hi @Mark Warren​ ,

thank you very much for your help.

with four parameters and without json it is much easier.

 

Best Regards Gerhard

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.