Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve performance of SortPreservingMergeExec #655

Closed
NGA-TRAN opened this issue Jul 2, 2021 · 1 comment · Fixed by #722
Closed

Improve performance of SortPreservingMergeExec #655

NGA-TRAN opened this issue Jul 2, 2021 · 1 comment · Fixed by #722
Labels
enhancement New feature or request

Comments

@NGA-TRAN
Copy link
Contributor

NGA-TRAN commented Jul 2, 2021 •
edited
Loading

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
While doing experience in IOx, we have discovered that SortPreservingMergeExec is expensive and very CPU bound. The issues are fully described here

Describe the solution you'd like
The SortPreservingMergeExec is known the be sub-optimal when it was implemented and at least 3 improvements have been identified inside the code. This ticket is to improve those and further if any.
@NGA-TRAN NGA-TRAN added the enhancement New feature or request label Jul 2, 2021
@alamb
Copy link
Contributor

alamb commented Jul 2, 2021

I believe @e-dard has plans to look into this ticket

@e-dard e-dard mentioned this issue Jul 6, 2021
Merged
@e-dard e-dard mentioned this issue Jul 13, 2021
Merged
andygrove added a commit to andygrove/datafusion that referenced this issue Jan 31, 2025
@andygrove
chore: Refactoring of CometError/SparkError (apache#655)
2f22a4d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

perf: improve performance of SortPreservingMergeExec operator e-dard/arrow-datafusion
2 participants
@alamb @NGA-TRAN