Skip to content

Update evaluators to consider NaN for evaluating upper/lower bounds  #1761

@yyanyy

Description

@yyanyy

See discussion and the test case, due to the comparison implementation of ORC stats, for float/double columns in ORC files, if the first value in a file is NaN, metrics of this file will report NaN for both upper and lower bound despite that the column could contain non-NaN data. This means that today we may unexpectedly skip including an ORC file for predicates that utilize bounds. We need to update evaluators to consider this edge case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions