Skip to main content
Open

Writing custom key-value metadata to Parquet

Related products:Integrations
  • August 15, 2022
  • 1 reply
  • 120 views

nampreetatsafe
Safer
Forum|alt.badge.img+14

Add the ability to write custom key-value metadata to the output parquet files.

Request originated from this Forums post.

1 reply

aegman
Contributor
Forum|alt.badge.img+3
  • Contributor
  • August 16, 2022

This is useful for any Parquet file, but my primary use case from my original question was trying to read/write GeoParquet.

Its possible to use the PythonCaller in combination with the Parquet reader to read GeoParquet (FYI, its slow!).

But with the writer, I could create GeoParquet files using python, but they throw column length mismatch errors when trying to read with the FME Parquet reader. Using the FME Parquet writer, it can't write the metadata, and I couldn't successfully update the created file with the metadata.

 

Any plans for GeoParquet support?