New Reader/Writer: HDF5

For a project we will have to work with the HDF5 Format. It would be nice if this format will be supported.

Maybe we can do something with this? http://www.h5py.org/

Global Precipitation Measurement (GPM) is produced and archived in HDF5. It would be great to be able to work with HDF5 in FME.

Can I use python library for creating hdf5 transformer? If yes than how?

Any update on if there is a plan to support HDF5 in FME?

No update at this moment I'm afraid. It is still planned, but I'm afraid I have no idea when it might be implemented. Certainly not for 2019. Sorry. If you have an urgent need and a solid business case then let your sales contact at Safe know. It's one way that we might increase the priority. At the moment it's set about as high as I can control.

Ok thanks for the update mark!

NOAA is providing navigation data including, water levels, winds, currents and wave heights in HDF5 format. It would be very helpful to be able to read this data in FME.

ATLAS/ICESat-2 Level 3A land and vegetation height data is provided as HDF5 formatted files. Having FME support this data format would be very useful.

Adding to the noise to increase the priority. The O&G industry needs it.

Now an OGC standard:

https://www.ogc.org/pressroom/pressreleases/3226?utm_source=phplist747&utm_medium=email&utm_content=HTML&utm_campaign=OGC+adopts+and+publishes+Hierarchical+Data+Format+Version+5+%28HDF5%29+Core+Standard

I would like to be able to use it for GEDI LIDAR data

The mining & oil/gas industries are adopting open source software, similar to QGIS

for oil/gas the standards are =

HDF5 https://www.hdfgroup.org/solutions/hdf5/

RESQML for reservoir characteristics & earth models e.g https://www.energistics.org/resqml-standards/

for mining/mineral exploration =

OMF e.g. seequent https://www.seequent.com/the-open-mining-format/

GEOH5 e.g. https://geoh5py.readthedocs.io/en/latest/content/geoh5_format/index.html & https://mirageoscience.com/mining-industry-software/geoscience-analyst/

should FME create reader/writer support for these, it would help greatly with 3D visualisation collaborations

New standards for ENCs (especially S-102) will be using hdf5 (https://www.admiralty.co.uk/access-data/s-100).

Hdf5 support in FME will be appreciated!

We need this added as a precursor to adding support for the 'KEA' GDAL driver (https://gdal.org/drivers/raster/kea.html) which we use for storing satellite imagery and model data.

Obviously for at least 8 long years now, the FME user community has been longing for HDF5 readers and writers, FOR EIGHT YEARS!
Is there still no better solution than laboriously working with some Python plugins?
We urgently need a simple and powerful solution to read and calculate with huge hourly HDF5 meteorology raster data.

HDF5 for oil/gas industry will be great to finally see

Can Safe also look deep into supporting OMF e.g. seequent https://www.seequent.com/the-open-mining-format/ - 

Hi Everyone!

Thank you all for your patience regarding this format… We are finally looking into adding a reader and writer for HDF5 into FME.

There are already good leads in this thread but I’d love to better understand how you’re using HDF5 and how you’d use it in FME workflows. Could you share a bit more about your use case(s)? For example:

1. What type of data are you working with in HDF5 (ex: simulations, environmental models, imagery)?
2. Do you need to read, write, or both?
3. Are there specific features that matter most to you (ex: chunking, compression, metadata, or handling of multi-dimensional datasets)?
4. Could you provide a few sample files or representative datasets (if possible - you can DM me files if you’re more comfortable)?

This information will help us shape our approach and identify the most important requirements for FME. Thank you again!