Skip to main content

WorkspacePictureHello, I am new to streaming using FME, and have a few basic questions about how to make it work.

 

I am looking to use FME to stream data from a GNSS receiver. Data is produced as an XML stream. This requires filtering to get the necessary features I am interested in and then I write it to a CSV and ultimately upload to Microsoft Azure.

 

I can get the basic Web>XML>CSV>Azure process to work, but I'm struggling with the time-looping aspects. The incoming data is continuous - I want to be able to package 1-minutes worth of data into a csv, then 15-minutes worth of csvs into a folder to be uploaded into Azure (to cut down on discreet uploads).

 

I'm not sure where to even start, although the time windower and workspace runner transformers look promising? I will have access to FME Server to automate the process.

 

As a secondary point, does anyone know if there is a simple way to tweak readers to iterate the same process for different source URLs? We have 20+ receivers, each with a different IP address, but the URL strings and data structure they produce are otherwise the same. I suspect I may have to set up 20+ different workspaces or flowlines within one workspace, but if there is a more efficient system, that would be good to know about!

 

I've uploaded a copy of my workspace.

Be the first to reply!

Reply