-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Zed can't read Parquet output from DuckDB #4527
Comments
This was referenced Apr 21, 2023
nwt
added a commit
that referenced
this issue
Apr 25, 2023
Reading and writing are much faster with it than with github.com/fraugster/parquet-go. Its only apparent drawback is that it offers no easy way to support Zed's duration and float16 types, and writing a value containing either produces a cryptic error. $ echo '{a:1.(float16)}' | zq -f parquet - parquetio: unsupported type: not implemented yet Closes #764, closes #4278, and closes #4527.
Verified in Zed commit deea4a4. The Parquet format that's output by DuckDB is now readable.
Thanks @nwt! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Repro is with Zed commit 0a375f8. The test data originally came from a community user that reported brimdata/zui#2751 and brimdata/zui#2754.
Start with the attached test data imdb.csv.
A user imports it into DuckDB as a table and then exports that table as Parquet.
Zed cannot read the Parquet that was produced.
However, tools like https://parquetreader.com seem able to read it without complaint.
I know we've suspected that following through on #4278 may allow us to cover more Parquet dialects, so I'll mark this issue as dependent on that one just in case that alone fixes it.
The text was updated successfully, but these errors were encountered: