Open
Connecting to parquet files in SQLExecutor/SQLCreator (using DuckDB Reader?)
When I got a humongous pile of parquet-files I had to find a way to read and filter the data in them. I found this and it works great, https://duckdb.org/docs/guides/import/parquet_import.html.Fast SQL-queries on all files without having to move the data into something else first!Using DBeaver and a DuckDB connection ":memory:" I can do queries like this:select count(distinct Id) FROM "D:\myfiles\**\*.parquet"where startingdate >= '2020-07-27 00:00:00'
Perhaps this will be a bonus with the upcoming DuckDB-reader/writer...??? :)
Migration Note: Idea originally posted on 2024-02-01
Be the first to reply!
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.