Solved

Lat/long data from WebSocketReceive can not be spatialized into a database

  • 2 November 2018
  • 8 replies
  • 2 views
aguan
Rank
Badge +4

My 1st time to use web socket data stream service. I use WebSocketReceiver to connect to a wss service. After using a JSONExtractor toextract the needed key/value pairs, I can write the data into an Oracle non-spatial table. The data has latitude/longitude, so I add a VertexCreator to create a point. But as I connect the VertexCreator to an Oracle SDE or File Geodatabase feature class, and run the workspace, the streaming goes on, but nothing happens in the writer, the feature class in SDE or File GeoDB is not created. No errors. I am pretty sure the lat/long values are valid. Any clues?

icon

Best answer by fmelizard 3 November 2018, 20:05

View original

8 replies

nathanatsafe
Rank
Badge +7

Hi @ag thanks for your post.

It sounds like this should be working in FME. If you inspect your features after the VertexCreator, before they enter your writer, do they look like they have the correct geometry? Have you ensured that the JSON is parsed correctly?

If you can share your workspace and a small set of sample data that reproduces the problem, that might help troubleshooting.

 

Hope this helps!

Nathan

aguan
Rank
Badge +4

Hi @ag thanks for your post.

It sounds like this should be working in FME. If you inspect your features after the VertexCreator, before they enter your writer, do they look like they have the correct geometry? Have you ensured that the JSON is parsed correctly?

If you can share your workspace and a small set of sample data that reproduces the problem, that might help troubleshooting.

 

Hope this helps!

Nathan

@NathanAtSafe See attached workspace. You can try writing to a feature class in file geodatabase if you don't have access to SDE. I also tried creating non-spatial table in file geodatabase, but not working. I only succeed in creating non=spatial Oracle table. The FME version is 2018.1.0.1

nathanatsafe
Rank
Badge +7

@NathanAtSafe See attached workspace. You can try writing to a feature class in file geodatabase if you don't have access to SDE. I also tried creating non-spatial table in file geodatabase, but not working. I only succeed in creating non=spatial Oracle table. The FME version is 2018.1.0.1

@ag Thanks for the follow up. This seems to be working for me https://www.screencast.com/t/rEWugFoH2L. If you delete the Oracle and SDE writer does just a File Geodatabase writer work? One thing to note is that if the Features Per Transaction parameter on your writer is set to 1000 you'll need to allow 1000 features to flow through the workspace for it to commit a transaction. You could lower this (as low as 1) if you wish to write fewer than 1000 features.

Let me know if you're still stuck.

 

 

Nathan
aguan
Rank
Badge +4

@ag Thanks for the follow up. This seems to be working for me https://www.screencast.com/t/rEWugFoH2L. If you delete the Oracle and SDE writer does just a File Geodatabase writer work? One thing to note is that if the Features Per Transaction parameter on your writer is set to 1000 you'll need to allow 1000 features to flow through the workspace for it to commit a transaction. You could lower this (as low as 1) if you wish to write fewer than 1000 features.

Let me know if you're still stuck.

 

 

Nathan

@NathanAtSafe. This is really Odd. After I deleted the Oracle Non-Spatial and ArcSDE writers, it wrote the spatial to the file geodatabase. I tried only keeping the ArcSDE writer, no luck. The VESSEL_FENCE_SPATIAL is not created. Can you test the arcSDE writer?

nathanatsafe
Rank
Badge +7

@NathanAtSafe. This is really Odd. After I deleted the Oracle Non-Spatial and ArcSDE writers, it wrote the spatial to the file geodatabase. I tried only keeping the ArcSDE writer, no luck. The VESSEL_FENCE_SPATIAL is not created. Can you test the arcSDE writer?

@ag SDE is working for me, as is writing to both File Geodatabase and SDE Feature Classes in the same workspace. Can you test with a simple workspace to make sure that SDE is working fine with FME on your machine?

aguan
Rank
Badge +4

@ag SDE is working for me, as is writing to both File Geodatabase and SDE Feature Classes in the same workspace. Can you test with a simple workspace to make sure that SDE is working fine with FME on your machine?

@NathanAtSafe. This is really odd. I am using FME with SDE everyday, and never saw something seems going into the SDE writer feature type, but nothing actually committed. Will look into more. Thanks for the quick testing.

fmelizard
Rank
Userlevel 4
Badge +13
There's good news and bad news. The bad news is: due to a design decision of 20+ years ago, when you use multiple writers in a workspace, only the first writer will write data UNTIL the input shuts down and the workspace run ends. Then the other writers will, one at a time, write any data that was destined to them. We've had a long running request to change this and it is near top of list, but not yet underway. I apologize that this still hasn't been dealt with.

 

So the behaviour you're seeing is entirely how FME works. Your workspace connects to a stream and will never end really, and therefore, only the first writer will ever get data.

So what is to be done? That's where the good news kicks in. If you use FeatureWriter transformers instead of traditional writers, then the problem goes away. FeatureWriter transformers are all active all the time (unless you do dataset fanout but I don't think that is your case). Suggest you replace all 3 of your writers with FeatureWriters. Do let us know the outcome.

 

aguan
Rank
Badge +4
There's good news and bad news. The bad news is: due to a design decision of 20+ years ago, when you use multiple writers in a workspace, only the first writer will write data UNTIL the input shuts down and the workspace run ends. Then the other writers will, one at a time, write any data that was destined to them. We've had a long running request to change this and it is near top of list, but not yet underway. I apologize that this still hasn't been dealt with.

 

So the behaviour you're seeing is entirely how FME works. Your workspace connects to a stream and will never end really, and therefore, only the first writer will ever get data.

So what is to be done? That's where the good news kicks in. If you use FeatureWriter transformers instead of traditional writers, then the problem goes away. FeatureWriter transformers are all active all the time (unless you do dataset fanout but I don't think that is your case). Suggest you replace all 3 of your writers with FeatureWriters. Do let us know the outcome.

 

@dalestsafe That explains it. When I had the non-spatial Oracle writer as the 1st added writer, it wrote to it and not the SDE, although it appeared writing to SDE. After I remove the other writers, the Oracle SDE writer gives an Oracle error, which was attributed to be one field named "long" which is invalid. After I changed it to be "longitude", it wrote to SDE successfully. I actually only need SDE writer so it works for me. I need to learn more about the Web Socket Transformer, like streaming to ESRI Portal web map or Geoevent server. Thanks for your help.

Reply


Community Stats

28,694
Posts
109,434
Replies
36,544
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.