Skip to content

Use SpillManager in AggregateExec and SortMergeJoinExec #15374

@2010YOUY01

Description

@2010YOUY01

Is your feature request related to a problem or challenge?

A new interface for spilling intermediate RecordBatches is proposed in #15355, it can be used in aggregate/SMJ's spilling logic.

Describe the solution you'd like

  1. Look for occurrences of the following 3 functions, and replace with the equivalent function in SpillManager
    pub(crate) fn read_spill_as_stream(

    pub(crate) fn spill_record_batches(

    pub fn spill_record_batch_by_size(
  2. Remove (those with pub(crate)) and deprecate (with pub) the above 3 functions

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions