Skip to main content
Question

Read datetimes from parquet files with FeatureReader transformer

  • September 21, 2023
  • 3 replies
  • 53 views

erburu
Participant
Forum|alt.badge.img

I am trying to read a parquet file with the "FeatureReader" transformer and I have no problems doing so. However, the three columns that I have in my parquet file, which are datetimes [ns], do not appear in the parquet. If I use an "Attribute Exposer" I can't get them either, they seem to come as "missing". Any alternative in this regard?

 

3 replies

virtualcitymatt
Celebrity
Forum|alt.badge.img+35

Which version of FME are you working with? Have you tired it with the latest 2023.1?


debbiatsafe
Safer
Forum|alt.badge.img+20
  • Safer
  • September 22, 2023

Hello @erburu​ 

I agree with @virtualcitymatt​'s suggestion of trying to read the Parquet file with the latest FME 2023.1 available here.

To give some background, read support for the deprecated int96 timestamp data type was recently added to the Parquet format. You should see the following warning "PARQUET Reader: Attribute '<Timestamp>' type is unsupported and column will not be read" when trying to read data stored in the deprecated data type. Previous versions of the Parquet reader only supported timestamps that is an int64 number as defined in the format specifications.


erburu
Participant
Forum|alt.badge.img
  • Author
  • Participant
  • September 25, 2023

I am with version 2022, it seems that in version 2023.1 the problem is solved.

 

Thank you very much


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings