Solved

How do I reliably connect FME Server 2018/19 to ArcGIS Online?

P
Badge

I'm having mixed results with ArcGIS Online connections on a new install of FME Server 2019.

I can publish a workbench with a ArcGIS Online connection (publish the web connection also) and it will work on FME Server 2019 immediately. Then after a little while it will stop connecting an AGOL will start rejecting connections. I'm assuming the workbench generated a token and the token expired???

Under my Web Connections on FME Server I have a ESRI ArcGIS Online connection which says:

Esri ArcGIS Online Web Service has not been configured for Web authorization. Jobs will run with your published credentials but you will not be able to authorize or create new Esri ArcGIS Online Web Connections until Esri ArcGIS Online Web Service has been configured in "Manage Web Services".

 

More Information

I go through the process of making an application on ArcGIS Online getting the app id and the client secret and the redirect uri is prefilled with something like - urn:ietf:wg:oauth:2.0:oob

I Manage Web Services - ESRI ArcGIS Online - and fill in the details above.

I then get

"You can Authorize at any time to verify this connection will work " and a working Authorize button which lets me login into ArcGIS Online and gives me OAuth2 Approval.

Please copy this code, switch to your application and paste it there:

Where do I copy that code?

The workbench doesn't connect to AGOL from FME Server now, saying www.arcgis.com is rejecting my connection.

Our server is a private server and not visible outside our network. Doesn't have SSL capability (at the moment).

Can someone point to me to clear documentation? I've seen a lot of different discussions and mixed results.

icon

Best answer by siennaatsafe 22 March 2019, 19:41

View original

17 replies

siennaatsafe
Rank
Badge +9

Hello @peterkt,

Thank you for posting your question to the community. I believe your issue is not setting the re-direct URI to <yourhostname>/fmeoauth. Then, using that same re-direct URI in your web connection. I've outlined the full instructions below for anyone else that may need this information however I believe you may only need steps 7-9. For now, I believe having a private network is okay with ArcGIS online. However, this may change if ArcGIS online changes something on their end. If they do, these are the instructions you can follow.

 

  1. Create a workspace with an ArcGIS online connection and publish it to FME Server. Please ensure you upload the connection. 2019-03-22-11-18-19
  2. Go to the Web Connections page in FME Server and select manage web connections. Click on the ArcGIS online web connection to edit it. 2019-03-22-11-22-33
  3. In a new tab go to your ArcGIS online account and create an app. In ArcGIS Online go to Content > My Content > Add Item. Select Application. Then select Application again and give your App a title and tags.
  4. Once you have an app. Click on it and select Settings.
  5. Once you scroll down the page you should see Registered Info. Click Registered Info to view the App information.
  6. Copy and Paste the Client ID and Secret to your FME Server Page2019-03-22-11-28-12
  7. Click Update to add your FME Server Redirect Uri which should be <yourServerHotst>/fmeoauth2019-03-22-11-31-00
  8. Click Add and then Update
  9. Paste that same Redirect Uri into FME Server
  10. Click OK to save the web connection
  11. Then, click on Web Connections again
  12. Select your ArcGIS online Web Connection
  13. Select Authorize 2019-03-22-11-36-17
  14. This will pop up and ArcGIS login page. Fill in your ArcGIS Online login credentials and select Sign In.
  15. You should see a dialog that your connection was successful. Click OK and try running your job.

If you follow these steps and still have an error, there may be a network issue preventing your successful connection. If that is the case please open a case here https://www.safe.com/support/report-a-problem/

P
Badge

Hi,

Thanks for the info.

It has to be a problem with the redirect.

I have an application on AGOL and and I copy client id and secret and don't have a redirect it works for half an hour in scheduled tasks until, I guess, the token expires.

If I fill in the redirect I get and go to authorize and login it gives me the FME Server OAuth Service FAILED.

It I don't use http://kedshrfme02/fmeoauth and use something like the default urn:ietf:wg:oauth:2.0:oob it does authorize (see 4th picture below), but still doesn't work when I run the workbench.

Could it be that our server needs direct connection to the internet?

We have a proxy which Desktop has a setting for and I've set the proxy settings for Server which appear to be successful since other Readers that read files down from websites are working.

 

 

siennaatsafe
Rank
Badge +9

Hi,

Thanks for the info.

It has to be a problem with the redirect.

I have an application on AGOL and and I copy client id and secret and don't have a redirect it works for half an hour in scheduled tasks until, I guess, the token expires.

If I fill in the redirect I get and go to authorize and login it gives me the FME Server OAuth Service FAILED.

It I don't use http://kedshrfme02/fmeoauth and use something like the default urn:ietf:wg:oauth:2.0:oob it does authorize (see 4th picture below), but still doesn't work when I run the workbench.

Could it be that our server needs direct connection to the internet?

We have a proxy which Desktop has a setting for and I've set the proxy settings for Server which appear to be successful since other Readers that read files down from websites are working.

 

 

Hello @peterkt,

In FME Server we aren't currently supporting proxies like we are on FME Desktop. We are hoping to improve our support in FME Server 2019.1. However, we do have a patch available that should get you up and running if you are using an un-authenticated proxy. Please open a support case and we will assist you in getting your web connection validated https://www.safe.com/support/report-a-problem/

 

 

Sorry for any inconvenience this may have caused.

 

Kind regards,

Sienna

P
Badge

Hello @peterkt,

In FME Server we aren't currently supporting proxies like we are on FME Desktop. We are hoping to improve our support in FME Server 2019.1. However, we do have a patch available that should get you up and running if you are using an un-authenticated proxy. Please open a support case and we will assist you in getting your web connection validated https://www.safe.com/support/report-a-problem/

 

 

Sorry for any inconvenience this may have caused.

 

Kind regards,

Sienna

Thanks Sienna for clarifying. Our proxy is authenticated to top things off. So my only hope is that I can work on getting a proxy bypass with our security team.

I thought proxy authentication was handled in Server as I'd set the authenticated proxy up in the settings.

Any chance 2019.1 will handle authenticated proxies?

siennaatsafe
Rank
Badge +9

Thanks Sienna for clarifying. Our proxy is authenticated to top things off. So my only hope is that I can work on getting a proxy bypass with our security team.

I thought proxy authentication was handled in Server as I'd set the authenticated proxy up in the settings.

Any chance 2019.1 will handle authenticated proxies?

Hello @peterkt,

I'm sorry we aren't supporting authenticated proxies at the moment. To clarify if you are using an authenticated proxy, everything involving the engines will work fine. Therefore, you can run jobs, and it should be okay. Anything involving the core will not work, this is mainly just an issue if you are using web connections.

We are looking into supporting authenticated proxies. We just needed to determine if there was a demand for it. With your case, it does look like there is a demand for it. I've escalated your case internally and I'll be updating this thread when I can confirm whether it will be in 2019.1.

Kind regards,

Sienna

 

 

****Update***** Authenticated proxies are now supported in 2019.1
tomf
Rank
Badge +7

Hi Sienna,

Any news on authenticated proxies in 2019.1 yet?

Cheers, Tom

 

siennaatsafe
Rank
Badge +9

Hi Sienna,

Any news on authenticated proxies in 2019.1 yet?

Cheers, Tom

 

Hi @tomf,

Yes! As of build 19608, we have support for authenticated proxies and a new interface to set the proxy.

P
Badge

Hi @tomf,

Yes! As of build 19608, we have support for authenticated proxies and a new interface to set the proxy.

Hi Sienna,

I can confirm from our end ArcGIS Online connections work through a secure firewall.

We have installed 2019.1 on our production.

I've set up our proxy settings in the new web interface.

I had to go back and make sure the web app settings from AGOL were correct and authorize via the manage web connections step.

The AGOL connected workbench works without any modifications.

Cheers to Safe for getting this work.

PS it appears to also fix a lot of our Secure Proxy authentication issues - double yay.

Peter

siennaatsafe
Rank
Badge +9

Hi Sienna,

I can confirm from our end ArcGIS Online connections work through a secure firewall.

We have installed 2019.1 on our production.

I've set up our proxy settings in the new web interface.

I had to go back and make sure the web app settings from AGOL were correct and authorize via the manage web connections step.

The AGOL connected workbench works without any modifications.

Cheers to Safe for getting this work.

PS it appears to also fix a lot of our Secure Proxy authentication issues - double yay.

Peter

@peterkt That's great to hear! I passed on your comments to the developers :)

andrewspencer
Badge

Hello @peterkt,

I'm sorry we aren't supporting authenticated proxies at the moment. To clarify if you are using an authenticated proxy, everything involving the engines will work fine. Therefore, you can run jobs, and it should be okay. Anything involving the core will not work, this is mainly just an issue if you are using web connections.

We are looking into supporting authenticated proxies. We just needed to determine if there was a demand for it. With your case, it does look like there is a demand for it. I've escalated your case internally and I'll be updating this thread when I can confirm whether it will be in 2019.1.

Kind regards,

Sienna

 

 

****Update***** Authenticated proxies are now supported in 2019.1

Hi Sienna, Is there any update to this. We're having a similar issue - using the AGOL Reader (and presumably, Writer, though we don't get that far) we fail with a:

ArcGIS Online Feature Service Reader: Requesting portal information

Error: QSInvalidRequest

Works every time on Desktop, and for a short while on server. But server schedules are useless.

FME Server goes through our proxy, and proxy authentication has been set up correctly as far as I can tell - scheduled web downloads and publishing to CKAN work fine.

In the meantime, I've found that the Portal Reader and Writer work well with AGOL and use token security, though I suppose this won't be a permanent state of affairs.

We're still on Server 2017 by the way - but this is key for us so if a fix is coming we'll wait....

Thanks

Andrew

 

 

siennaatsafe
Rank
Badge +9

Hi Sienna, Is there any update to this. We're having a similar issue - using the AGOL Reader (and presumably, Writer, though we don't get that far) we fail with a:

ArcGIS Online Feature Service Reader: Requesting portal information

Error: QSInvalidRequest

Works every time on Desktop, and for a short while on server. But server schedules are useless.

FME Server goes through our proxy, and proxy authentication has been set up correctly as far as I can tell - scheduled web downloads and publishing to CKAN work fine.

In the meantime, I've found that the Portal Reader and Writer work well with AGOL and use token security, though I suppose this won't be a permanent state of affairs.

We're still on Server 2017 by the way - but this is key for us so if a fix is coming we'll wait....

Thanks

Andrew

 

 

Hello Andrew,

That does sound like a proxy issue which is solved in FME Server 2019.1. We now supporting authorized proxies. By upgrading your Server you should be able to use the AGOL reader and writer as normal. You can download FME Server from this link: https://www.safe.com/support/support-resources/fme-downloads/

Kind regards,

Sienna

mattwilkie
Rank
Badge +8

What is the new doc page for the AGOL "create an app" step? The existing url( https://doc.arcgis.com/en/marketplace/provider/create-listing.htm) redirects to https://doc.arcgis.com/en/marketplace/help/become-a-provider.htm

siennaatsafe
Rank
Badge +9

What is the new doc page for the AGOL "create an app" step? The existing url( https://doc.arcgis.com/en/marketplace/provider/create-listing.htm) redirects to https://doc.arcgis.com/en/marketplace/help/become-a-provider.htm

Hi @mattw1ilkie​ ,

 

Sorry about that. I'll see if I can find a new resource. In the meantime I'll write out the instructions I follow.

  1. Go to ArcGIS Online and select the Content tab
  2. Select Add Item 2020-10-06_15-02-44
  3. Select An Application 2020-10-06_15-04-05
  4. For Type select Application and fill out the parameters (it doesn't matter to FME Server what the title or the tags are). Then click Add Item and you will have an application.2020-10-06_15-04-27
mattwilkie
Rank
Badge +8

Hi @mattw1ilkie​ ,

 

Sorry about that. I'll see if I can find a new resource. In the meantime I'll write out the instructions I follow.

  1. Go to ArcGIS Online and select the Content tab
  2. Select Add Item 2020-10-06_15-02-44
  3. Select An Application 2020-10-06_15-04-05
  4. For Type select Application and fill out the parameters (it doesn't matter to FME Server what the title or the tags are). Then click Add Item and you will have an application.2020-10-06_15-04-27

Thanks Sienna. It took a bit to wrap my ahead around the fact that the "app" we're creating is only to authenticate and not do anything else. Once that was out of the way I could get through the next few steps.

 

Now I'm stuck with "FME Server OAuth Service FAILED" when going to https://server/fmeoauth. I'll leave troubleshooting that until tomorrow.

 

And, heads up for a future question: how to support more than one AGOL account?

philippeb
Badge +8

Hello @peterkt,

Thank you for posting your question to the community. I believe your issue is not setting the re-direct URI to <yourhostname>/fmeoauth. Then, using that same re-direct URI in your web connection. I've outlined the full instructions below for anyone else that may need this information however I believe you may only need steps 7-9. For now, I believe having a private network is okay with ArcGIS online. However, this may change if ArcGIS online changes something on their end. If they do, these are the instructions you can follow.

 

  1. Create a workspace with an ArcGIS online connection and publish it to FME Server. Please ensure you upload the connection. 2019-03-22-11-18-19
  2. Go to the Web Connections page in FME Server and select manage web connections. Click on the ArcGIS online web connection to edit it. 2019-03-22-11-22-33
  3. In a new tab go to your ArcGIS online account and create an app. In ArcGIS Online go to Content > My Content > Add Item. Select Application. Then select Application again and give your App a title and tags.
  4. Once you have an app. Click on it and select Settings.
  5. Once you scroll down the page you should see Registered Info. Click Registered Info to view the App information.
  6. Copy and Paste the Client ID and Secret to your FME Server Page2019-03-22-11-28-12
  7. Click Update to add your FME Server Redirect Uri which should be <yourServerHotst>/fmeoauth2019-03-22-11-31-00
  8. Click Add and then Update
  9. Paste that same Redirect Uri into FME Server
  10. Click OK to save the web connection
  11. Then, click on Web Connections again
  12. Select your ArcGIS online Web Connection
  13. Select Authorize 2019-03-22-11-36-17
  14. This will pop up and ArcGIS login page. Fill in your ArcGIS Online login credentials and select Sign In.
  15. You should see a dialog that your connection was successful. Click OK and try running your job.

If you follow these steps and still have an error, there may be a network issue preventing your successful connection. If that is the case please open a case here https://www.safe.com/support/report-a-problem/

Is that normal that your link  https://doc.arcgis.com/en/marketplace/provider/create-listing.htm redirects me automatically to this https://www.esri.com/en-us/arcgis-marketplace/become-provider" alt="http://https://www.esri.com/en-us/arcgis-marketplace/become-provider" target="_blank">https://www.esri.com/en-us/arcgis-marketplace/become-provider ?

In FME Server 2020, when I click on Add an app to ArcGIS Online I'm redirected to the same place...

 

I have to authorize again my ArcGIS Online account on FME Server 2020 but it seems a little bit more complex than on FME Server 2017, where I only had to click on the Authorize button. Maybe it's only my understanding that is not optimal yet though. I migrated to 2020 for a week only.

 

Thanks !

philippeb
Badge +8

Is that normal that your link  https://doc.arcgis.com/en/marketplace/provider/create-listing.htm redirects me automatically to this https://www.esri.com/en-us/arcgis-marketplace/become-provider" alt="http://https://www.esri.com/en-us/arcgis-marketplace/become-provider" target="_blank">https://www.esri.com/en-us/arcgis-marketplace/become-provider ?

In FME Server 2020, when I click on Add an app to ArcGIS Online I'm redirected to the same place...

 

I have to authorize again my ArcGIS Online account on FME Server 2020 but it seems a little bit more complex than on FME Server 2017, where I only had to click on the Authorize button. Maybe it's only my understanding that is not optimal yet though. I migrated to 2020 for a week only.

 

Thanks !

OK never mind. I just found out that I did the same thing for FME Server 2017! My bad...

Everything is working perfectly fine now on my FME Server 2020 :)

Thanks for this useful tuto!!

siennaatsafe
Rank
Badge +9

Is that normal that your link  https://doc.arcgis.com/en/marketplace/provider/create-listing.htm redirects me automatically to this https://www.esri.com/en-us/arcgis-marketplace/become-provider" alt="http://https://www.esri.com/en-us/arcgis-marketplace/become-provider" target="_blank">https://www.esri.com/en-us/arcgis-marketplace/become-provider ?

In FME Server 2020, when I click on Add an app to ArcGIS Online I'm redirected to the same place...

 

I have to authorize again my ArcGIS Online account on FME Server 2020 but it seems a little bit more complex than on FME Server 2017, where I only had to click on the Authorize button. Maybe it's only my understanding that is not optimal yet though. I migrated to 2020 for a week only.

 

Thanks !

Haha that's great to know!

 

I'll update the links! It looks like the ones I had are now broken.

Reply


Community Stats

28,729
Posts
109,527
Replies
36,565
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.