Solved

Topic Monitoring not working FME Server 2017

  • 13 February 2018
  • 16 replies
  • 25 views
mattenp
Rank

Hi,

if I click on "Topic monitoring" in the fme server portal I get the error:

"There was an error connecting to monitoring service"

The port 7078 is open and if I request http://fmedev-test.de:7078/, I get the answer "not a WebSocket handshake request: missing upgrade". So the firewall rules should be right and a telnet command to fme server via 7078 also works.

I've also set WEB_SOCKET_SERVER_NAME in

Utilities\\tomcat\\webapps\\fmeserver\\WEB-INF\\conf\\propertiesFile.properties

The logs from websocket.log in publishers and subscribers are:

Opening log file using character encoding UTF8

 

Tue-13-Feb-2018 02:45:58.721 PM INFORM main 801001 : Publisher "websocket" is starting up ...

 

Tue-13-Feb-2018 02:45:58.721 PM INFORM main 801002 : Publisher "websocket" is updating configurations ...

Opening log file using character encoding UTF8

 

Tue-13-Feb-2018 02:45:52.909 PM INFORM main 803001 : Subscriber "websocket" is starting up ...

 

Tue-13-Feb-2018 02:45:52.909 PM INFORM main 803002 : Subscriber "websocket" is updating configurations ...

Do you have any idea to solve the problem?

Best regards,

Matthias

icon

Best answer by jlutherthomas 20 July 2018, 21:01

View original

16 replies

philippeb
Badge +8

Restart your FME Server?

philippeb
Badge +8

Have you read this? topics

mattenp
Rank

hi, I have read this topics , but that's not the problem, we are not using https at the moment. Also a restart has been more often done.

Best regards,

Matthias

B
Hi Matthias,

 

 

Did you ever manage to solve this problem? I'm experiencing the same issue.

 

 

Kind regards,

 

Hillebrand

 

 

D

Basically the same problem here.

Do you use ARR with ISS (rewriting requests to another port, as per KB article Using IIS and ARR as a reverse proxy )?

We have this setup and FME server (version 2018) itself is working fine, it seems. Only when trying to monitor a topic we get the same error.

Requesting http://localhost:7078 (also the public host name http://fme:7078) gives the same message "not a WebSocket handshake request: missing upgrade", so the general setup seems to be OK.

 

BTW: Like the original thread opener we are not using https, and of course there were plenty of restarts and testing before this post.

Maybe meanwhile there are any ideas how to solve this issue?

 

Any help is welcome!

Best regards,

 

Andreas
J
Badge +2

Hi @mattenp and @digit42

 

Have you looked at our Notifications troubleshooting guide?

 

 

A few things you might want to try:

 

Does topic monitoring work on the same server that FME Server is installed on?
  • Test on the server, and a remote machine.

Do you see any errors in the Developer Information window (In Google Chrome right click > Inspect > Network > refresh page)? Try see what happens when you try to start topic monitoring as well.

 

 

Are you able to share the full websocket.log ?

 

 

Also I can confirm on a server with working Topic Monitoring, I see the "not a WebSocket handshake request: missing upgrade". I think this is because just going to that url isn't opening up a channel for websockets.

D

Hi @mattenp and @digit42

 

Have you looked at our Notifications troubleshooting guide?

 

 

A few things you might want to try:

 

Does topic monitoring work on the same server that FME Server is installed on?
  • Test on the server, and a remote machine.

Do you see any errors in the Developer Information window (In Google Chrome right click > Inspect > Network > refresh page)? Try see what happens when you try to start topic monitoring as well.

 

 

Are you able to share the full websocket.log ?

 

 

Also I can confirm on a server with working Topic Monitoring, I see the "not a WebSocket handshake request: missing upgrade". I think this is because just going to that url isn't opening up a channel for websockets.

Hi @jlutherthomas 

 

thank you for answering.

 

 

Yes, I did read the Notifcations troubleshooting guide but didn't find a solution for this particular problem.

 

 

Inspecting the page with developer tools shows a websocket request witch has status "(pending)" for a while and then "Finished", but Size is 0 Bytes and the time column still says "Pending". Initiator is "safe-software-app:18310.min.js:6".

 

Request URL is ws://fme.xxx.yyy.zzz.de:7078/websocket and the timing tab says: "Connection Start: Stalled" (Time 23:4 hrs). Request/Response is "Content Donwload" (Time 7.26 ms)

 

 

Until now I tested topic monitoring via the servers web interface only from a remote machine (the mentionened request to http://localhost:7078 was done on the server directly).

 

I did a test for topic monitoring via web interface with a browser directly on the server now. And guess what: I did not get an error message there! Topic monitoring works when using a browser directly on the server.

 

Thank you for that suggestion - I totally missed that obvious one.

 

 

So it seems that port 7078 requests are not coming through to remote hosts - but there are no firewall rules blocking that port. 

 

Do you have an idea where I should look to fix this?

 

 

I attached the complete fmewebsocket.log. The first lines were in there before my test on the server (i.e. where I got the error messages that don't show up in the log). The lines starting at Fr-20-Jul-2018 06:03:06.213 PM were added when I did the test directly on the server.

 

 

fmewebsocket-log.zip

 

 

The fmenotification.log is only 3 lines long and doesn't give any useful information here. It shows only entries from the time the tests failed:

 

 

Opening log file using character encoding UTF8
Fr-20-Jul-2018 05:12:19.253 PM   INFORM   localhost-startStop-7   422000 : FME Server Notification Service started in locale: de_DE
Fr-20-Jul-2018 05:12:19.253 PM   INFORM   localhost-startStop-7   422002 : Service default response format is: HTML

 

Best regards,

 

Andreas

 

J
Badge +2
On your FME Server, can you set an explicit firewall rule to accept 7078 incoming/outgoing and see if that changes anything? Are you working with an express install of FME Server?

 

 

If from a remote machine it's getting through to FME Server in anyway you should see in the websocket.log it starting up and opening a channel at that time, so it will be interesting to see.

 

In the developer information if it's working you should be able to see it successfully switch protocols (green) like this: https://www.dropbox.com/s/fik0ujkpuyzlltf/Screenshot%202018-07-20%2011.39.48.png?dl=0

 

 

I don't know whether monitoring network traffic is something you could do/try, to see where it's trying to go (the request) and with your IT work to unblock it?

 

J
Badge +2
Hi @jlutherthomas 

 

thank you for answering.

 

 

Yes, I did read the Notifcations troubleshooting guide but didn't find a solution for this particular problem.

 

 

Inspecting the page with developer tools shows a websocket request witch has status "(pending)" for a while and then "Finished", but Size is 0 Bytes and the time column still says "Pending". Initiator is "safe-software-app:18310.min.js:6".

 

Request URL is ws://fme.xxx.yyy.zzz.de:7078/websocket and the timing tab says: "Connection Start: Stalled" (Time 23:4 hrs). Request/Response is "Content Donwload" (Time 7.26 ms)

 

 

Until now I tested topic monitoring via the servers web interface only from a remote machine (the mentionened request to http://localhost:7078 was done on the server directly).

 

I did a test for topic monitoring via web interface with a browser directly on the server now. And guess what: I did not get an error message there! Topic monitoring works when using a browser directly on the server.

 

Thank you for that suggestion - I totally missed that obvious one.

 

 

So it seems that port 7078 requests are not coming through to remote hosts - but there are no firewall rules blocking that port. 

 

Do you have an idea where I should look to fix this?

 

 

I attached the complete fmewebsocket.log. The first lines were in there before my test on the server (i.e. where I got the error messages that don't show up in the log). The lines starting at Fr-20-Jul-2018 06:03:06.213 PM were added when I did the test directly on the server.

 

 

fmewebsocket-log.zip

 

 

The fmenotification.log is only 3 lines long and doesn't give any useful information here. It shows only entries from the time the tests failed:

 

 

Opening log file using character encoding UTF8
Fr-20-Jul-2018 05:12:19.253 PM   INFORM   localhost-startStop-7   422000 : FME Server Notification Service started in locale: de_DE
Fr-20-Jul-2018 05:12:19.253 PM   INFORM   localhost-startStop-7   422002 : Service default response format is: HTML

 

Best regards,

 

Andreas

 

On your FME Server, can you set an explicit firewall rule to accept 7078 incoming/outgoing and see if that changes anything? Are you working with an express install of FME Server?

 

 

If from a remote machine it's getting through to FME Server in anyway you should see in the websocket.log it starting up and opening a channel at that time, so it will be interesting to see.

 

In the developer information if it's working you should be able to see it successfully switch protocols (green) like this: https://www.dropbox.com/s/fik0ujkpuyzlltf/Screenshot%202018-07-20%2011.39.48.png?dl=0

 

 

I don't know whether monitoring network traffic is something you could do/try, to see where it's trying to go (the request) and with your IT work to unblock it?

 

D
On your FME Server, can you set an explicit firewall rule to accept 7078 incoming/outgoing and see if that changes anything? Are you working with an express install of FME Server?

 

 

If from a remote machine it's getting through to FME Server in anyway you should see in the websocket.log it starting up and opening a channel at that time, so it will be interesting to see.

 

In the developer information if it's working you should be able to see it successfully switch protocols (green) like this: https://www.dropbox.com/s/fik0ujkpuyzlltf/Screenshot%202018-07-20%2011.39.48.png?dl=0

 

 

I don't know whether monitoring network traffic is something you could do/try, to see where it's trying to go (the request) and with your IT work to unblock it?

 

It's working now!

 

I had to add an explicit firewall rule to accept incoming requests on port 7078. After doing that the error messages are gone and dev tools show "switch protocols" now like in your screenshot.

 

 

Thanks for your help - it is highly appreciated!

 

 

Best regards,

 

Andreas

 

J
Badge +1

@jlutherthomas did you ever find a solution to this problem?  I just upgraded FME server to 2018.1.1.1 (express install) and I'm getting the same error when attempting monitor topics remotely.  I get this error message in the chrome console, 

"WebSocket connection to 'ws://fme.mydomain.com:7078/websocket' failed: Connection closed before receiving a handshake response"   

Logging onto the machine and hitting the topic monitoring using localhost it is working correctly.  Windows firewall is turned off. Ideas?

 

 

J
Badge +2

@jlutherthomas did you ever find a solution to this problem?  I just upgraded FME server to 2018.1.1.1 (express install) and I'm getting the same error when attempting monitor topics remotely.  I get this error message in the chrome console, 

"WebSocket connection to 'ws://fme.mydomain.com:7078/websocket' failed: Connection closed before receiving a handshake response"   

Logging onto the machine and hitting the topic monitoring using localhost it is working correctly.  Windows firewall is turned off. Ideas?

 

 

Hi @justincornell

 

 

Is windows firewall turned off on the FME Server and the machine you're using to access it?

 

It sounds like it isn't managing to connect or it's taking too long. Do you see the attempts to connect from remote machines in the websocket.log file?
J
Badge +1

Hi @justincornell

 

 

Is windows firewall turned off on the FME Server and the machine you're using to access it?

 

It sounds like it isn't managing to connect or it's taking too long. Do you see the attempts to connect from remote machines in the websocket.log file?

Hi @jlutherthomas.  Windows firewall is turned off for the server.  I checked the websocket.log and there is nothing that gets put in the log when topic monitoring is started.  (under resources\logs\core\current\subscribers and resources\logs\core\current\publishers) It only shows entries like this below in the subscribers log.  I believe this entry is being added when an event is triggered.  I see no entries in the log when log in on localhost or try to connect from another machine.

 

subscribers\websocket.log

Mon-11-Feb-2019 03:08:00.765 PM   INFORM   main   803013 : WebSocket client connecting to host "localhost" on port "7078"
Mon-11-Feb-2019 03:08:00.812 PM   INFORM   New I/O worker #244   803006 : WebSocket client connected: -1310226486
Mon-11-Feb-2019 03:09:45.114 PM   INFORM   websocket-cache-1   803015 : WebSocket client sending close
Mon-11-Feb-2019 03:09:45.114 PM   INFORM   New I/O worker #244   803005 : Channel closed. WebSocket client disconnected: -1310226486

 

 

J
Badge +2

Hi @jlutherthomas.  Windows firewall is turned off for the server.  I checked the websocket.log and there is nothing that gets put in the log when topic monitoring is started.  (under resources\logs\core\current\subscribers and resources\logs\core\current\publishers) It only shows entries like this below in the subscribers log.  I believe this entry is being added when an event is triggered.  I see no entries in the log when log in on localhost or try to connect from another machine.

 

subscribers\websocket.log

Mon-11-Feb-2019 03:08:00.765 PM   INFORM   main   803013 : WebSocket client connecting to host "localhost" on port "7078"
Mon-11-Feb-2019 03:08:00.812 PM   INFORM   New I/O worker #244   803006 : WebSocket client connected: -1310226486
Mon-11-Feb-2019 03:09:45.114 PM   INFORM   websocket-cache-1   803015 : WebSocket client sending close
Mon-11-Feb-2019 03:09:45.114 PM   INFORM   New I/O worker #244   803005 : Channel closed. WebSocket client disconnected: -1310226486

 

 

Hi @justincornell

 

Even though the firewall is turned off, can you try adding an explicit firewall rule to accept incoming requests on port 7078 (incoming and outgoing on the FME server machine) and see if that works?

 

 

Have you checked to see what is going on in the developer tools window when you try to start topic monitoring remotely? Are there any errors or is it just pending?
G
Badge

Hi all,

 

I have the same problem.

 

My setup:

FME Server 2019.0.0.1 

 

Build 19246 - win64 

 

Using full version of FME server.

On FME server machine port 7078 is explicitly added to the firewall for in/out connections

Websocket is enabled in IIS in FME server

 

Errors I get:

on FME server localhost:7078 I get 

not a WebSocket handshake request: missing upgrade

In FME server interface on the topic monitoring page I get 

There was an error connecting to monitoring service 

 

From another machines I can not access fme-server-url:7078, getting 

This site can’t be reached

 

Firewall has 7078 exception on the local machine.

 

Any ideas?

R

Hi All, same error here - have not been resolved yet - it also impacts jobsubmiiter

Reply


Community Stats

28,724
Posts
109,518
Replies
36,564
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.