Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add custom "fused" groupby aggregation to Dask cuDF #17009

Merged
merged 13 commits into from
Oct 18, 2024
Merged

Add custom "fused" groupby aggregation to Dask cuDF #17009

merged 13 commits into from
Oct 18, 2024

Conversation

rjzamora
Copy link
Member

@rjzamora rjzamora commented Oct 7, 2024 •
edited
Loading

Description

The legacy Dask cuDF implementation uses a custom code path for GroupBy aggregations. However, when query-planning is enabled (the default), we use the same algorithm as the pandas backend. This PR ports the custom "fused aggregation" code path over to the dask-expr version of Dask cuDF.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@rjzamora rjzamora self-assigned this Oct 7, 2024
@github-actions github-actions bot added the Python Affects Python cuDF API. label Oct 7, 2024
@rjzamora rjzamora added 2 - In Progress Currently a work in progress dask Dask issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 8, 2024
@rjzamora rjzamora marked this pull request as ready for review October 9, 2024 14:29
@rjzamora rjzamora requested a review from a team as a code owner October 9, 2024 14:29
@rjzamora rjzamora added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Oct 9, 2024
@rjzamora rjzamora changed the title [WIP] Add custom groupby-aggregation to Dask cuDF Add custom "fused" groupby aggregation to Dask cuDF Oct 9, 2024
wence-
wence- approved these changes Oct 15, 2024
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To the best of my understanding this makes sense. Could we add a test please?

@rjzamora
Copy link
Member Author

Could we add a test please?

Thanks for the review @wence- ! I updated test_groupby_agg_params to cover the fused=False case now that fused=True is already tested everywhere else.

@rjzamora rjzamora added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Oct 18, 2024
@rjzamora
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 6ad9074 into rapidsai:branch-24.12 Oct 18, 2024
98 checks passed
@rjzamora rjzamora deleted the custom-groupby-dask-cudf branch October 18, 2024 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wence- wence- wence- approved these changes

@madsbk madsbk Awaiting requested review from madsbk

Assignees

@rjzamora rjzamora

Labels
5 - Ready to Merge Testing and reviews complete, ready to merge dask Dask issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@rjzamora @wence-