I'm looking into different storage stategies for storing IoT-data in our Data lake (ADLS Gen2) and i would very much like to be able to read data from the Data lake and into FME for both analysis and transform-operations. We have more or less decided to store our cleaned IoT sensor data as AVRO since it is such an optimized storage format. It is also possible that we will store Parquet-files for specific use cases but for now the focus is on AVRO. However, i cannot seem to find an easy way to consume AVRO/Parquet in FME apart from struggling with AVRO modules in Python etc. It would be neat to have an AVRO and Parquet reader since they are such common big data formats.
Page 1 / 1
Parquet is now supported as of FME 2021.0.
For years AVRO has to me been one of the Dutch broadcasting companies. But, recently I encounter it more often as a data-format in very interesting applications.
Would be great if Safe gets it on the roadmap for FME.