I'm looking into different storage stategies for storing IoT-data in our Data lake (ADLS Gen2) and i would very much like to be able to read data from the Data lake and into FME for both analysis and transform-operations. We have more or less decided to store our cleaned IoT sensor data as AVRO since it is such an optimized storage format. It is also possible that we will store Parquet-files for specific use cases but for now the focus is on AVRO. However, i cannot seem to find an easy way to consume AVRO/Parquet in FME apart from struggling with AVRO modules in Python etc. It would be neat to have an AVRO and Parquet reader since they are such common big data formats.
<strong>This post is closed to further activity.</strong><br />
It may be a question with a best answer, an implemented idea, or just a post needing no comment.<br />
If you have a follow-up or related question, please <a href="https://community.safe.com/topic/new">post a new question or idea</a>.<br />
If there is a genuine update to be made, please contact us and request that the post is reopened.
+1Parquet is now supported as of FME 2021.0.
+18- Supporter
For years AVRO has to me been one of the Dutch broadcasting companies. But, recently I encounter it more often as a data-format in very interesting applications.
Would be great if Safe gets it on the roadmap for FME.
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.