Skip to content

Improve performance of ScalarValue::to_array_of_size() for Lists #18177

@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

Follow on to

In this PR, @2010YOUY01 added special case logic to the NestedLoopsJoin
operator because ScalarValue::to_array_of_size() was very inefficient when
used on DataType::Utf8View values.

However, the logic is applicable to all uses of ScalarValue::to_array_of_size() not
only in NestedLoopsJoin.

Describe the solution you'd like

No response

Describe alternatives you've considered

This ticket tracks moving the special case into ScalarValue::to_array_of_size() itself rather than
a special case in NestedLoopsJoin.

Additional context

The root cause of why this is slow is here:

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceMake DataFusion faster

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions