Backwards-incompatible change to thrift metadata

[asfimport]

Sometime in the last few versions, a isStructOrUnion field has been added to the thrift.descriptor written to the parquet header:

{
    "children": [ ... ],
    "id": "STRUCT", 
    "structOrUnionType": "STRUCT"
}

The current release now throws an exception when that field is missing / UNKNOWN). This makes it impossible to read back thrift data written using a previous release.

Reporter: Ben Kirwin

Related issues:

• ThriftSchemaConverter throws for unknown struct or union type (duplicates)

_{Note: This issue was originally created as PARQUET-405. Please see the migration documentation for further details.}

[asfimport]

Ben Kirwin:
Failure stacktrace:

Caused by: org.apache.parquet.ShouldNeverHappenException: Encountered UNKNOWN StructOrUnionType
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.isUnion(ThriftSchemaConvertVisitor.java:337)
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:213)
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:76)
    at org.apache.parquet.thrift.struct.ThriftType$StructType.accept(ThriftType.java:272)
    at org.apache.parquet.thrift.ThriftSchemaConvertVisitor.convert(ThriftSchemaConvertVisitor.java:88)
    at org.apache.parquet.thrift.ThriftSchemaConverter.convert(ThriftSchemaConverter.java:60)
    at org.apache.parquet.thrift.ThriftRecordConverter.<init>(ThriftRecordConverter.java:797)
    at org.apache.parquet.scrooge.ScroogeRecordConverter.<init>(ScroogeRecordConverter.java:36)

[asfimport]

Ryan Blue / @rdblue:
Thanks, Ben! Both for reporting the issue and for helping us keep the issues organized.