Parquet Thrift should discard records with unrecognized union members #1756
Description
When writing parquet-thrift files, when a thrift record with an unknown union member is encountered, it should be considered a bad record and discarded. Currently, because unions are treated as structs with one optional field per union member, parquet-thrift happily writes the empty struct, but then crashes in the read path when trying to read this record.
We should discard these records in the write path, just as we discard other unparseable records.
Reporter: Alex Levenson / @isnotinvain
Note: This issue was originally created as PARQUET-215. Please see the migration documentation for further details.