Skip to content

Parquet reader cannot handle files with unknown logical types #8776

@etseidl

Description

@etseidl

Describe the bug
The logical type annotation in the Parquet schema is meant to be forward compatible. Readers encountering a value they do not recognize should still be able to read the file using just the physical type.

To Reproduce
Try reading the file unknown-logical-type.parquet in the parquet-testing repo

% ~/src/arrow-rs/target/debug/parquet-read unknown-logical-type.parquet 

thread 'main' panicked at parquet/src/bin/parquet-read.rs:79:50:
Failed to create reader: General("Cannot annotate _Unknown { field_id: 2555 } from BYTE_ARRAY for field 'column with unknown type'")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Expected behavior
File should be read

Additional context
The thrift remodel allowed for forward compatibility of logical types, but there remain a few places in the code that cannot handle them.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions