Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand PartialSort usage in more queries to reduce memory usage #9153

Open
ahmetenis opened this issue Feb 7, 2024 · 0 comments
Open

Expand PartialSort usage in more queries to reduce memory usage #9153

ahmetenis opened this issue Feb 7, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@ahmetenis
Copy link
Contributor

ahmetenis commented Feb 7, 2024

Is your feature request related to a problem or challenge?

PartialSort proposed in #9125 can also be used on bounded input which will help reducing memory usage for the sorts when the input is already partially sorted (input plan ordering and plan required ordering has a common prefix)

Describe the solution you'd like

We might need to consider incorporating ExternalSorter in PartialSortExec plan as it is used in SortExec. Then it should be completely safe to replace SortExec with PartialSortExec in EnforceSorting without checking for unboundedness of input.

Describe alternatives you've considered

Implementing partial sort logic inside SortExec and making SortExec choose which sorting logic to run might be an option.

It can also be argued that it is easier to understand the intention when PartialSortExec and SortExec are kept as separate plans.

Additional context

#7456

@ahmetenis ahmetenis added the enhancement New feature or request label Feb 7, 2024
@alamb alamb changed the title Expand PartialSort usage to bounded input cases to reduce memory usage Expand PartialSort usage in more queries to reduce memory usage Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant