Skip to main content
Question

Writing IoT Data Stream to ArcGIS Online Feature Service via FME

  • October 24, 2025
  • 5 replies
  • 97 views

tej
Contributor
Forum|alt.badge.img+2

Hello everyone,

I’m currently facing a challenge with writing an IoT data stream directly to an ArcGIS Online (AGOL) hosted feature service using FME. While I’ve successfully configured the stream to write to a local SQLite database, I’ve been unable to achieve the same with AGOL.

I’ve attempted to use the Esri ArcGIS Feature Service Writer transformer, but unfortunately, it hasn’t worked as expected in stream mode. I’m wondering if there’s an alternative transformer or method that supports continuous writing to an AGOL feature service.

Here’s a brief overview of my FME workspace setup:

  • Input Source: Reading data from an MQTT broker in stream mode using the MQTTConnector.
  • Data Processing: Filtering incoming messages into different groups based on sensor types.
  • Output Goal: Writing the processed data to an AGOL hosted feature service.
  • Runtime: The workspace is designed to run continuously to handle real-time data.

Any guidance or suggestions on how to achieve this would be greatly appreciated. Thank you in advance!

5 replies

crystalatsafe
Safer
Forum|alt.badge.img+26

Hi ​@tej 
Can you describe the issue you're experiencing with the ArcGIS Online Feature Service writer? Are you seeing any errors? Is it not writing any features? 

Are you working with lower volume stream data (~a few features per minute) by chance? If you are working with lower volume stream data, it may not have generated enough features to trigger an AGOL write transaction. If this is the case, you can lower the threshold in the AGOL writer's Features Per Request parameter to write features more frequently.

 


tej
Contributor
Forum|alt.badge.img+2
  • Author
  • Contributor
  • November 4, 2025

Hi ​@crystalatsafe,

I'm currently unable to write any incoming data to either the AGOL-hosted feature service or the related table in AGOL.

Previously, I tried reducing the "features per request" setting to 1, but even with that adjustment, the transformer still fails to write the data.

Even in translation logs, it never had any logs about that transformer trying to write to AGOL

 


tej
Contributor
Forum|alt.badge.img+2
  • Author
  • Contributor
  • November 4, 2025

I was able to successfully upload data to the AGOL feature service when using the MQTT connector in batch mode. However, the data fails to write when I switch to stream mode, which is the mode I actually need for my workflow.

 

 

 


crystalatsafe
Safer
Forum|alt.badge.img+26

Hi ​@tej 

I’m sorry for the late response, I am still looking into this but I wanted to share a few articles that may be helpful. 

FME Form Tips for Working with Continuous Data Streams

Real-time Events and Systems

Was the screenshot of the log you shared with batch mode selected? 

Could you please send the log file with log debug (Tools → FME Options → Translation → check log debug), when you run the workspace with stream mode selected in the MQTT connector?


crystalatsafe
Safer
Forum|alt.badge.img+26

Hi ​@tej 

Are you using any blocking transformers? Since you mentioned grouping features based on types, you should check out: Process When Group Changes.  

You might need to use TimeWindower and heartbeat feature if the tips in that article didn't work.

If possible, please send your workspace along with sample data, and I would be happy to take a look.