Question

Web connection working in HTTP caller, inside a loop on FME Server - only for one day

  • 15 December 2022
  • 8 replies
  • 22 views
K
Badge

Hello,

 

I was using a HTTP caller with a web connection fine on FME Desktop and Server - This was running fine on a daily schedule. I then realized that the HTTP caller needed to run in a loop so I got it working on both Desktop and server fine.

 

The issue is, the workbench will only work for one day on the server, and then it returns a '401 Unauthorized' the next day. I find that if I re-publish the web connection, the workbench runs fine......until the next day. Does anyone know if this is a glitch or if there is some work around? The workbench was previously running fine until I added the loop and there hasn't been any changes to web connection each time I publish

 

Any help is welcome!

 

 

Many thanks,

 

Sam

8 replies

T
Badge +3

Hi Sam,

What kind of Web Connection are you using? One that uses a token by any chance?

K
Badge

Hi Sam,

What kind of Web Connection are you using? One that uses a token by any chance?

Hi @thijsknapen​ ,

 

To create the web connection I used the MS Sharepoint online template. It's OAuth2.0 and not a token service

T
Badge +3

Hi Sam,

What kind of Web Connection are you using? One that uses a token by any chance?

Hi @kiwi_sam​ ,

Check. Previously I also encountered some issues on FME Server when using OAuth2.0 Web Connection. Also OAuth2.0 uses a token, see e.g. the underlying settings for the connection;

imageWhat I seemed to notice from the log is that the refreshing of the token does work on FME Desktop, but not on FME Server. Maybe you can have a look in your workspace logs if this is the case.

N.b. I'm not sure if log debug mode is needed to see these messages.

 

K
Badge

Hi @kiwi_sam​ ,

Check. Previously I also encountered some issues on FME Server when using OAuth2.0 Web Connection. Also OAuth2.0 uses a token, see e.g. the underlying settings for the connection;

imageWhat I seemed to notice from the log is that the refreshing of the token does work on FME Desktop, but not on FME Server. Maybe you can have a look in your workspace logs if this is the case.

N.b. I'm not sure if log debug mode is needed to see these messages.

 

Ah, yes there is a refresh token parameter. Similar to yours but I have client secret and tenancy ID in there:

image 

I've just stumbled across a similar question someone had https://community.safe.com/s/question/0D54Q000080hMsoSAE/how-does-one-troubleshoot-oauth-token-refresh-issues-on-fme-server. It makes sense now why it's only working for the first day (Probably hour or so but I only run it once a day)

 

Did you manage to trouble shoot this at all in your environment, or is it still an outstanding issue?

K
Badge

Hi @kiwi_sam​ ,

Check. Previously I also encountered some issues on FME Server when using OAuth2.0 Web Connection. Also OAuth2.0 uses a token, see e.g. the underlying settings for the connection;

imageWhat I seemed to notice from the log is that the refreshing of the token does work on FME Desktop, but not on FME Server. Maybe you can have a look in your workspace logs if this is the case.

N.b. I'm not sure if log debug mode is needed to see these messages.

 

I've just noticed the manage web services area in the web connections. It states that the job can't authorize or create new credentials as it hasn't been configured. I now need to figure out what the redirect URI is. I'm using the same one as the web connection is in FME server but it FME Server can't connect to it:

imageimage 

I'll keep trying and post any solutions I find in here. Thanks for all of your help @thijsknapen​ 

T
Badge +3

Hi @kiwi_sam​ ,

Check. Previously I also encountered some issues on FME Server when using OAuth2.0 Web Connection. Also OAuth2.0 uses a token, see e.g. the underlying settings for the connection;

imageWhat I seemed to notice from the log is that the refreshing of the token does work on FME Desktop, but not on FME Server. Maybe you can have a look in your workspace logs if this is the case.

N.b. I'm not sure if log debug mode is needed to see these messages.

 

Hi @kiwi_sam​ ,

 

As I was using a local API, I wasn't able to create a reproducible issue report for Safe. I did mention it over live chat at the time, but it seemed like an unknown issue at that point in time.

 

I did store some elaborate logs though. Maybe you can check in your logs if you find the same.

On FME Desktop I found the following lines when the token would expire;

WARN |Current time: 1640181427 , Expiry time: 1640180677

INFORM|Updating access token for '<Classified>'

 

On FME Server I would only see the first line (and not the second line on 'Updating access token for ...');

WARN |Current time: 1640185381 , Expiry time: 1640185027

 

And a few lines beneath that FME Server log file I would find;

INFORM|HTTPCaller_3 (HTTPFactory): [1]: HTTP response header: HTTP/1.1 401 Unauthorized

...

INFORM|HTTPCaller_3 (HTTPFactory): [1]: HTTP response header: WWW-Authenticate: Bearer error="invalid_token", error_description="The token expired at '12/22/2021 14:57:08'"

 

Continuing the FME Server job log, I would then find a seeming retry attempt. I.e. shortly after a new warning log message 'WARN |Current time: X , Expiry time: Y', followed by response headers indicating the 'HTTP/1.1 401 Unauthorized' and later also the same WWW-authentication error ... error_description="The token expired at '12/22/2021 14:57:08'" (i.e. also same DateTime as before).

 

However, what I found interesting is that 'this second 'WARN |Current time: X , Expiry time: Y' message was as follows;

WARN |Current time: 1640185383 , Expiry time: 1640185382

Thus interestingly a new Expiry time as before, but (consistently...) a single second before the (updated) current time...

virtualcitymatt
Rank
Userlevel 4
Badge +26

https://community.safe.com/s/article/How-to-create-a-Sharepoint-Web-Connection

 

There's this pretty detailed tutorial about setting up a connection. It might be a bit out of date now though. did you have a look through here?

david_r
Rank
Userlevel 4

Hi @kiwi_sam​ ,

Check. Previously I also encountered some issues on FME Server when using OAuth2.0 Web Connection. Also OAuth2.0 uses a token, see e.g. the underlying settings for the connection;

imageWhat I seemed to notice from the log is that the refreshing of the token does work on FME Desktop, but not on FME Server. Maybe you can have a look in your workspace logs if this is the case.

N.b. I'm not sure if log debug mode is needed to see these messages.

 

The client ID, the client secret and the redirect URI are all defined by the Sharepoint admin when setting up the access. You have to make sure that you're using the exact same values in your FME web connection. It's usually not necessary that the redirect URI exists.

 

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.