Skip to main content
Open

Connecting to parquet files in SQLExecutor/SQLCreator (using DuckDB Reader?)

Related products:Integrations
danilo_fme
antoine
nordpil
nissedahlsten
josch
  • danilo_fme
    danilo_fme
  • antoine
    antoine
  • nordpil
    nordpil
  • nissedahlsten
    nissedahlsten
  • josch
    josch

nissedahlsten
Contributor
When I got a humongous pile of parquet-files I had to find a way to read and filter the data in them. I found this and it works great, https://duckdb.org/docs/guides/import/parquet_import.html.Fast SQL-queries on all files without having to move the data into something else first!Using DBeaver and a DuckDB connection ":memory:" I can do queries like this:select count(distinct Id) FROM "D:\myfiles\**\*.parquet"where startingdate >= '2020-07-27 00:00:00' Perhaps this will be a bonus with the upcoming DuckDB-reader/writer...??? :) Migration Note: Idea originally posted on 2024-02-01

0 replies

Be the first to reply!

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings