Skip to main content

Hello,

Situation:

I have 2 cameras for crowdcontrol. Measures people (walking, cycling) going in/out a certain area. Those cameras generate a stream that I can access through FME.

 

This stream is in this format:

jsonexample

When the stream is complete (or the timewindow is reached) we get an EOS (End Of Stream) thus the WebSocket connection can be closed. Is that possible?

Because when connection is still open it does not write the output to a Writer. Database/Plain text file/.... I can only see the output with a logger.

 

 

Most database writers have the option to set the transaction size. For live data you can set this to be one, you should see committed transactions for each feature.

Great question about closing the Connection though. I'm not sure if it's possible but you could use a Terminator transformer to terminate the process when it gets the EOS message. It's not very graceful because it will look like the process has failed, but you can customise the termination message to read EOS received or something.


Most database writers have the option to set the transaction size. For live data you can set this to be one, you should see committed transactions for each feature.

Great question about closing the Connection though. I'm not sure if it's possible but you could use a Terminator transformer to terminate the process when it gets the EOS message. It's not very graceful because it will look like the process has failed, but you can customise the termination message to read EOS received or something.

From the doc (if you are using this transformer) it sounds like if the host closes the connection then the transformer should 'finish' .

http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/websocketreceiver.htm

I haven't tested it though


From the doc (if you are using this transformer) it sounds like if the host closes the connection then the transformer should 'finish' .

http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/websocketreceiver.htm

I haven't tested it though

Their websocket API states that you should close the connection and since the WebSocketReceiver keeps running I think they aren't closing it :)

But I'm having a meeting with the developers to clear this out. It's quite a complex system: simplest put

  1. Authentication sign in,
  2. Setup Websocket connection,
  3. Channel configuration (setting the API enablement and data selection time range),
  4. Data streaming start ( Stream: true),
  5. Data stream stop (Stream: false)) (optional)
  6. Close WebSocket connection, or perform other queries and then close,
  7. Authentication sign out.

The queries are done by sending JSON queries with a WebSocketSender.

This is quite a fun project to do but the amount of grey hairs have doubled.


Most database writers have the option to set the transaction size. For live data you can set this to be one, you should see committed transactions for each feature.

Great question about closing the Connection though. I'm not sure if it's possible but you could use a Terminator transformer to terminate the process when it gets the EOS message. It's not very graceful because it will look like the process has failed, but you can customise the termination message to read EOS received or something.

So continuous writing to the DB is solved with the SQLExecutor transformer. Hooray!


So continuous writing to the DB is solved with the SQLExecutor transformer. Hooray!

Yeah! that would do it too! nice.


From the doc (if you are using this transformer) it sounds like if the host closes the connection then the transformer should 'finish' .

http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/websocketreceiver.htm

I haven't tested it though

Another potential option to look at is FME Server. If you have FME Server it can also be set up to receive Websocket streams. Another tool which you could look into is 'Node-RED'. a node.js based event handling system. I think this coupled with FME and FME Server would be a pretty effective system.

Something to look into for the future perhaps if you start getting even more grey hairs 😁


From the doc (if you are using this transformer) it sounds like if the host closes the connection then the transformer should 'finish' .

http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/websocketreceiver.htm

I haven't tested it though

We have an FME server but I would need 4 engines. I have 2 camera's that generate 4 streams (In and Out)..could be quite expensive 🙈


From the doc (if you are using this transformer) it sounds like if the host closes the connection then the transformer should 'finish' .

http://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Transformers/websocketreceiver.htm

I haven't tested it though

Ahh yeah for sure


Reply