Skip to content

Add option to disable summary metadata aggregation after MR jobs #1594

@asfimport

Description

@asfimport

The summary metadata flies produced at the end of MR jobs aren't required for the job to complete successfully or for reading the data, so this should be an optional step in the MR commit phase. I'd like to add a property parquet.enable.summary-metadata that turns this off if set to false. The default will be true / enabled to match the current behavior.

Reporter: Ryan Blue / @rdblue
Assignee: Ryan Blue / @rdblue

Note: This issue was originally created as PARQUET-107. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions