Skip to content

Support Union data type coercion #18825

@friendlymatthew

Description

@friendlymatthew

Is your feature request related to a problem or challenge?

Union arrays currently cannot be used in comparison operations with scalar values or other types. When attempting to filter or compare a union column against a constant value (e.g., where union_column = 67), datafusion fails during type coercion

This prevents common filtering and selection ops on union-typed data. At work, we interact with a lot of JSON and accessing data with -> returns Union types. Currently the following query will err:

select * from records
where json_val->'id' = 123

Describe the solution you'd like

  • Write a union_coercion function that handles union-to-scalar and union-to-union type coercion, following the pattern established by other composite types like struct_coercion and map_coercion
  • Extend comparison_coercion to chain to union_coercion as part of its coercion attempts

On the arrow-side, comparison kernel support for unions are being developed:

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions