Skip to content

Allow clients to control the GenericData object that is used to read Avro records #1672

@asfimport

Description

@asfimport

Right now, Parquet always uses the default SpecificData instance (retrieved by SpecificData.get()) to lookup the schemas for SpecificRecord subclasses. Unfortunately, if the definition of the SpecificRecord subclass is not available to the classloader used in SpecificData.get(), we will fail to find the definition of the SpecificRecord subclass and will fall back to returning a GenericRecord, which will cause a ClassCastException in any client code that is expecting an instance of the SpecificRecord subclass.

We can fix this limitation by allowing the client code to specify how to construct a custom instance of SpecificData (or any other subclass of GenericData) for Parquet to use, including instances of SpecificData that use alternative classloaders.

Reporter: Josh Wills / @jwills
Assignee: Josh Wills / @jwills

Note: This issue was originally created as PARQUET-140. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions