-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Description
ThriftSchemaConverter should either only be called on ThriftStruct's that have populated structOrUnionType metadata, or should support a mode where this data is unknown w/o throwing an exception.
Currently it is called using the file's metadata here:
https://github.com/apache/parquet-mr/blob/d6f082b9be5d507ff60c6bc83a179cc44015ab97/parquet-thrift/src/main/java/org/apache/parquet/thrift/ThriftRecordConverter.java#L797
One workaround is not not use the file matadata here but rather the schema from the thrift class. The other is to support unknown struct or union types
Reporter: Alex Levenson / @isnotinvain
Assignee: Alex Levenson / @isnotinvain
Related issues:
- Backwards-incompatible change to thrift metadata (is duplicated by)
PRs and other links:
Note: This issue was originally created as PARQUET-346. Please see the migration documentation for further details.