-
Notifications
You must be signed in to change notification settings - Fork 1k
Closed
Labels
Description
Describe the bug
The logical type annotation in the Parquet schema is meant to be forward compatible. Readers encountering a value they do not recognize should still be able to read the file using just the physical type.
To Reproduce
Try reading the file unknown-logical-type.parquet in the parquet-testing repo
% ~/src/arrow-rs/target/debug/parquet-read unknown-logical-type.parquet
thread 'main' panicked at parquet/src/bin/parquet-read.rs:79:50:
Failed to create reader: General("Cannot annotate _Unknown { field_id: 2555 } from BYTE_ARRAY for field 'column with unknown type'")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Expected behavior
File should be read
Additional context
The thrift remodel allowed for forward compatibility of logical types, but there remain a few places in the code that cannot handle them.
kylebarron