Skip to main content
Solved

I have a web service that I want to access with FME. How can I do that?

  • September 30, 2016
  • 3 replies
  • 184 views
fmelizard
Safer
Forum|alt.badge.img+22

I have a web service that I want to access with FME. How can I do that? There is something about REST and OAUTH and such. help?

Best answer by fmelizard

FME can absolutely connect to web services and has built in support for a number of popular ones.

If we don't have built in support, you can add it with this basic pattern.

Documentation for the Third Party Web Service you want to use
  • Locate the web service docs.
  • Locate the part that talks about creating an app or web service access or something like that.
  • Locate the part that talks about OAuth.
  • Locate the part that talks about REST API.

What do you want to do? Examples:
  • I want to download a table or a layer
  • I want to create a record
Determine the REST end-point required for your objective.If that doesn’t exist, then there is no point proceeding. You’ll need:
  • Request URL
  • HTTP method
  • HTTP parameters (optional)
  • HTTP query string (optional)
  • HTTP body (optional)
Often you are looking to work on a specific entity (e.g. table, row). You’ll need to figure out how that entity is referenced in the REST call (e.g. id, name) and then determining where that specific reference can be located (e.g. in the web app). This reference is then provided as part of the Request URL or sometimes the HTTP query string or HTTP body.

Determine the specific authentication implementation This is the hardest part. You’ll need to search the Third Party web service docs for the following information. You may need help from the company providing the service to find this information:
  • Client Id
  • Client Secret
  • Authorization URL
  • Access Token URL
  • Access Token Request format
There is another one that is a bit of a mystery:
  • Client Redirect URI
We tend to set that to the following value, which works in many cases: There may be even more things required, but these are the minimum. Create the web service authentication definition Next you need to create the definition in FME.
  • Open the preferences/options >
  • Web Connections >
  • Manage Services >
  • Scroll to the bottom of the page >
  • click the the plus button >
  • select OAuth 2.0 Service
  • Fill it out the information you recorded earlier

Finally, add the transformer that does the thing you actually want to do
  • Add an HTTPCaller transformer
  • Add the REST call information you recorded earlier.

Good luck.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

3 replies

fmelizard
Safer
Forum|alt.badge.img+22
  • fmelizardSafer
  • Author
  • Safer
  • Best Answer
  • September 30, 2016

FME can absolutely connect to web services and has built in support for a number of popular ones.

If we don't have built in support, you can add it with this basic pattern.

Documentation for the Third Party Web Service you want to use
  • Locate the web service docs.
  • Locate the part that talks about creating an app or web service access or something like that.
  • Locate the part that talks about OAuth.
  • Locate the part that talks about REST API.

What do you want to do? Examples:
  • I want to download a table or a layer
  • I want to create a record
Determine the REST end-point required for your objective.If that doesn’t exist, then there is no point proceeding. You’ll need:
  • Request URL
  • HTTP method
  • HTTP parameters (optional)
  • HTTP query string (optional)
  • HTTP body (optional)
Often you are looking to work on a specific entity (e.g. table, row). You’ll need to figure out how that entity is referenced in the REST call (e.g. id, name) and then determining where that specific reference can be located (e.g. in the web app). This reference is then provided as part of the Request URL or sometimes the HTTP query string or HTTP body.

Determine the specific authentication implementation This is the hardest part. You’ll need to search the Third Party web service docs for the following information. You may need help from the company providing the service to find this information:
  • Client Id
  • Client Secret
  • Authorization URL
  • Access Token URL
  • Access Token Request format
There is another one that is a bit of a mystery:
  • Client Redirect URI
We tend to set that to the following value, which works in many cases: There may be even more things required, but these are the minimum. Create the web service authentication definition Next you need to create the definition in FME.
  • Open the preferences/options >
  • Web Connections >
  • Manage Services >
  • Scroll to the bottom of the page >
  • click the the plus button >
  • select OAuth 2.0 Service
  • Fill it out the information you recorded earlier

Finally, add the transformer that does the thing you actually want to do
  • Add an HTTPCaller transformer
  • Add the REST call information you recorded earlier.

Good luck.

mark2atsafe
Safer
Forum|alt.badge.img+59
  • mark2atsafeSafer
  • Safer
  • March 23, 2017

Here's an example using Fitbit (note: I haven't gotten it to work yet, but I seem to have the right info in the right place).

1) I went to dev.fitbit.com and registered my "app":

0684Q00000ArJjZQAV.png

2) Fitbit accepted this and returned all the secrets I needed to know (some info truncated for privacy reasons):

0684Q00000ArKgQQAV.png

3) I opened Tools > FME Options > Web Connections > Manage Services and created a new service. I filled in the info given to me by Fitbit. I knew it was an Oauth2 connection, so I created one of those:

0684Q00000ArKT9QAN.png

Notice all I did was to paste in the values that were given by Fitbit.

4) It didn't work! I visited the api docs for Fitbit and from their example...

https://www.fitbit.com/oauth2/authorize?response_type=token&client;_id=22942C&redirect;_uri=http%3A%2F%2Fexample.com%2Ffitbit_auth&scope;=activity%20nutrition%20heartrate%20location%20nutrition%20profile%20settings%20sleep%20social%20weight&expires;_in=604800

...figured out that the paths they had given me were the truth, but not the whole truth. I had to edit the FME Authorization Parameters to include the type of authorization (code or token) and the Fitbit parts I wanted access to (Activity, Exercise, Nutrition, Sleep, etc).

 

I also checked the format for the refresh parameter and found it wasn't quite the default. I had to change the Request Format in FME to:

grant_type=refresh_token&refresh;_token=[REFRESH_TOKEN]&expires;_in=604800

Now the test button opened a web page and let me log in to Fitbit. 

Unfortunately that's where the progress ends... it works in a web browser but not in FME. I don't know if they are returning the token/code in a way that FME can't interpret.

Anyway... if it's of help I would say the steps to add a web service are...

  • Go to the web site and sign up. Most of them will give you the user ID and secret when you have signed up and registered an app.
  • Copy the secret info they give you into an FME web service definition.
  • Check the documentation. The info you are given might not include the arguments that FME has to put into the request.
  • Hit the Test button to see if it works! If it fails, see if you can paste the authentication URL into a web browser. If it won't work in a browser, then it won't work in FME.
  • If your authentication URL works in a web browser, but not in FME, let us know!

     

FME Evangelist to the Rich and Famous!!
jatoxa
Contributor
Forum|alt.badge.img+12
  • jatoxaContributor
  • Contributor
  • September 20, 2018

Hi Mark2AtSafe,

 

 

I know this is an old thread but I'm having some issues with oauth2 that I can't seem to fix.

 

I've made another thread about it here https://knowledge.safe.com/questions/77895/stored-web-connection.html

 

 

The first time I authenticated my profile when testing the web connection it asked me to login on the web service pop-up. However when I updated the credentials to another user it simply asks whether I will allow/deny the application but it never asks for login credentials again and it seems to cache the initial login credentials somewhere?

 

So on every PC where have logged in once I can't update the connection. I can even fill in random strings in the web connection parameters and it still retrieves my initial account info if I call it through the HTTP caller.

 

 

Also the API I want to connect needs proper use of the state parameter and I wonder how I would fill in a random string in the web connection setup?

 

 

The setup for the API is explained here in detail https://service.projectplace.com/apidocs/#articles/pageOAuth2.html

Community Stats

32,751
Posts
124,208
Replies
41,260
Members
Terms & ConditionsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.