Question

fme 2019 server fault tolerant setup for netscaler

R

Hi

I have setup fme server 2019, as a fault tolerant on 2 server, with share and a SQL Server Database in HA cluster

We are using netscaler for load balancing, but the documentation is not very clear on how to config the load balancer part

I am struggling to get the topic monitoring working for fault tolerant, and IT also unsure how to configure

our load balncer has been setup as http://gisfmelb:8080, and it opens up the fme server GUI interface, this seems to work OK.

but configuring the monitoring on port 7078 for websocket, we can't seem to get it right.

i am receiving this error on topic monitoring service

 

i was not sure, on the url you need for topic monitoring, setup as - ws://gisfmelb:7078/websocket

attached are the netscaler config settings, for review

 

 

 

 

 

 

 

1 Attachment

7 replies

steveatsafe
Rank
Badge +11

Hi @rudy_v

I've not worked with NetScaler LB so I can't say if the configuration is correct. Thank you for sharing the file and I'd love to see it when we get this working for you.

Where are you setting the URL? Are you doing it here: http://<yourserver>/fmeserver/#/configuration/services.

I have used HAProxy and it can check the health of the WebSocket and redirect traffic.

Did you see this documentation:

https://docs.safe.com/fme/html/FME_Server_Documentation/AdminGuide/Set-Up-Load-Balancer.htm

The requirement is this:

Always route traffic on port 7078 to one single WebSocket Server, but switch to use other WebSocket Server as backup if the WebSocket Server in use has failed. If the load balancer in use does not support automatic failover to backup servers, then customers will need manual intervention to change the load balancer routing to a healthy WebSocket Server.

 

 

SteveatSafe
R

So far we have tried everything we can, in my opion the documentation on this is very limited, and very hard to troubleshoot

we also receive this message:

173FMEServerJobSubmitter (ServerFactory): http://gisfmelb.8080 - Running workspace 'ArcHubCount_2Sql.fmw' in repository 'Migration'...174FMEServerJobSubmitter (ServerFactory): http://gisfmelb.8080 - Failed to submit request to run workspace 'ArcHubCount_2Sql.fmw' in repository 'Migration'175FMEServerJobSubmitter (ServerFactory): Reason - user 'admin' is not authorized to perform this action176The below feature caused the translation to be terminated

 

very strange that admin which are the superuser, comes up with this error.

I also cannot find in the log files why the topic monitoring fails connecting

Yes i have done the settings in the correct spots services and topic monitoring

 

My question was: ws://gisfmelb:7078/websocket is this the correct address in topic monitoring for fault tolerant, as this is nowhere documentated

 

We uninstalled the fault tolerant as it is giving too much issues and reverted back to standalone server installation as we had for 2012 to 2018 installations.

todd_davis
Rank
Userlevel 1
Badge +12

So far we have tried everything we can, in my opion the documentation on this is very limited, and very hard to troubleshoot

we also receive this message:

173FMEServerJobSubmitter (ServerFactory): http://gisfmelb.8080 - Running workspace 'ArcHubCount_2Sql.fmw' in repository 'Migration'...174FMEServerJobSubmitter (ServerFactory): http://gisfmelb.8080 - Failed to submit request to run workspace 'ArcHubCount_2Sql.fmw' in repository 'Migration'175FMEServerJobSubmitter (ServerFactory): Reason - user 'admin' is not authorized to perform this action176The below feature caused the translation to be terminated

 

very strange that admin which are the superuser, comes up with this error.

I also cannot find in the log files why the topic monitoring fails connecting

Yes i have done the settings in the correct spots services and topic monitoring

 

My question was: ws://gisfmelb:7078/websocket is this the correct address in topic monitoring for fault tolerant, as this is nowhere documentated

 

We uninstalled the fault tolerant as it is giving too much issues and reverted back to standalone server installation as we had for 2012 to 2018 installations.

If you setup your FME Server as gisfmelib on port 8080, your websocket will be ws://gisfmelb:7078/websocket. I emphasis this is your FME Server and not your netscaler.

Have you also tested that you can access ws://gisfmelb:7078/websocket using a web socket client (chrome or firefox extensions). You should be able to test it is working without topic monitoring working. Often port 7078 can be blocked, so its a good test to get out of the way. Once you are sure you have access, then you can work on topic monitoring. As Steve mentioned the documentation is there, but you may only specific aspects to implement, and thats going to come down to your network and setup. And I have found in our specific setup that an Imap topic monitor won't work, but other will (although we also have external wss reverting to ws internally).

For instance our setup under FME Server is to goto “System Configurations” “Networking” "Topic Monitoring" , set the Internal Websocket URL to ws://<url>:7078/websocket

And then we alter some of the configuration in FME Server:

· On the server goto <fme server share>\\localization\\publishers\\websocket\\websocket and change the URL to point at same location as previous step.

· Do the same for <fme server share>\\localization\\subscribers\\websocket\\websocket

· run C:\\Program Files\\FMEServer\\Clients\\utilities\\addPublishers.bat as administrator

· run C:\\Program Files\\FMEServer\\Clients\\utilities\\addPublishers.bat as administrator

· Restart FME Server

R
Badge

Hi Rudy

Have you had any progress on this? We have the exact same problem only we are using a F5 load balancer and the managers of those devices are unsure how to configure for web socket. The fault tolerant environment is working perfectly in 2017 without the load balancer (web application server using the old architecture). Everything seems to work on the new load balanced servers (2019) except the notifications. What is interesting is that the notifications to the server run OK when triggered from the workbench on a desktop but it fails when the same workbench is published to the server (wont run on the server). It even runs correctly when published and run on the older fme server and sent to the new servers - that is the new server won't talk to itself but will accept external requests to the notification service

Robert

DPFEM Hobart

steveatsafe
Rank
Badge +11

Hi Rudy

Have you had any progress on this? We have the exact same problem only we are using a F5 load balancer and the managers of those devices are unsure how to configure for web socket. The fault tolerant environment is working perfectly in 2017 without the load balancer (web application server using the old architecture). Everything seems to work on the new load balanced servers (2019) except the notifications. What is interesting is that the notifications to the server run OK when triggered from the workbench on a desktop but it fails when the same workbench is published to the server (wont run on the server). It even runs correctly when published and run on the older fme server and sent to the new servers - that is the new server won't talk to itself but will accept external requests to the notification service

Robert

DPFEM Hobart

Hi @rob_datavision,

I'm sorry you are running into this issue. I can't suggest anything here that would help you but I can tell you others have had success with f5 LB and FME Server.

Could you possibly submit a case so we can connect directly with you and see if we can decipher why this isn't working for you? www.safe.com/support - report a problem. You can address this KB in the case so we know you are the same person.

Cheers,

Steve

SteveatSafe
R

Hi Rudy

Have you had any progress on this? We have the exact same problem only we are using a F5 load balancer and the managers of those devices are unsure how to configure for web socket. The fault tolerant environment is working perfectly in 2017 without the load balancer (web application server using the old architecture). Everything seems to work on the new load balanced servers (2019) except the notifications. What is interesting is that the notifications to the server run OK when triggered from the workbench on a desktop but it fails when the same workbench is published to the server (wont run on the server). It even runs correctly when published and run on the older fme server and sent to the new servers - that is the new server won't talk to itself but will accept external requests to the notification service

Robert

DPFEM Hobart

Hi @rob_datavision

unfortunately no - our IT infrastructure can't get it to work. I am getting the same issue as you -

when i run a work bench on server with a fme server jobsubmitter it fails. We are getting in a Citrix consultant to come and check.

We are looking at other alternatives, which we can manage ourself - as we manage all the GIS and FME Servers, Desktop Software, GIS Web Developemnt and SQL Server Databases, it can be included in our stack.

R
Badge

Hi Rudy

Have you had any progress on this? We have the exact same problem only we are using a F5 load balancer and the managers of those devices are unsure how to configure for web socket. The fault tolerant environment is working perfectly in 2017 without the load balancer (web application server using the old architecture). Everything seems to work on the new load balanced servers (2019) except the notifications. What is interesting is that the notifications to the server run OK when triggered from the workbench on a desktop but it fails when the same workbench is published to the server (wont run on the server). It even runs correctly when published and run on the older fme server and sent to the new servers - that is the new server won't talk to itself but will accept external requests to the notification service

Robert

DPFEM Hobart

Hi Rudy

Sorry to hear you are still having problems.

Fortunately we have sorted out our problems.

Firstly we had not setup the F5 to cater for the websocket port. Once we did that the topic monitoring started working. But it did not solve our notification problem.

Last week Safe Software had a look at our problem and it turned out to be a proxy issue. Our site has multiple domains and the new proxy configuration in the admin settings of the fme server WebGUI does not support wildcard exceptions. We discovered this because the safe software technician also discovered the job submitter did not work. Once we disabled the proxy (on the admin page of the WebGUI) the job submitter and the notifications (for publish workspaces) started working. However, I realised later that fme server would then not communicate with external sites. Enable it again and the opposite was true.

What we had to do was disable proxy on the fme WebGUI and enable it using old method on the internet options for the fme service account, this allowed for exceptions (ignore proxy for internal etc). By doing this we managed to get everthing working correctly.

When you say the job submitter doesn't work it sounds like it was exactly the same problem. Safe Software were fantastic at helping me trouble shoot this.

 

These links provided by safe should help with the 2 proxy setting methods

Old Method

 

https://docs.safe.com/fme/html/FME_Server_Documentation/AdminGuide/Using_FME_Server_with_Proxy_Server.htm?Highlight=proxy

 

 

New Method

 

https://docs.safe.com/fme/html/FME_Server_Documentation/WebUI/Proxy.htm?Highlight=proxy

 

 

 

Reply


Community Stats

28,724
Posts
109,519
Replies
36,564
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.