Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pylibcudf.Scalar that interoperates with Arrow scalars #14133

Merged
merged 11 commits into from
Oct 6, 2023
Merged

Add pylibcudf.Scalar that interoperates with Arrow scalars #14133

merged 11 commits into from
Oct 6, 2023

Conversation

vyasr
Copy link
Contributor

@vyasr vyasr commented Sep 19, 2023

Description

This PR adds a new Scalar object to pylibcudf that will function as the pylibcudf equivalent of cudf::scalar. Unlike columns, which are typically operated on in the form of views rather than owning types by libcudf, owning scalars are accepted by (const) ref by libcudf APIs and no corresponding view type exists. Therefore, pylibcudf.Scalar differs from pylibcudf.Column by actually owning an instance of the underlying libcudf type (cudf::scalar). Construction of pylibcudf Scalars is expected to be done from an Arrow scalar.

This PR relies on #14124 and should not be merged until after that one.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@vyasr vyasr added feature request New feature or request 2 - In Progress Currently a work in progress Python Affects Python cuDF API. Cython non-breaking Non-breaking change labels Sep 19, 2023
@vyasr vyasr self-assigned this Sep 19, 2023
@github-actions github-actions bot added libcudf Affects libcudf (C++/CUDA) code. CMake CMake build issue labels Sep 19, 2023
@vyasr vyasr mentioned this pull request Sep 23, 2023
Merged
3 tasks
@github-actions github-actions bot removed the libcudf Affects libcudf (C++/CUDA) code. label Sep 25, 2023
@vyasr vyasr marked this pull request as ready for review September 25, 2023 22:56
@vyasr vyasr requested a review from a team as a code owner September 25, 2023 22:56
@vyasr vyasr requested review from wence- and isVoid September 25, 2023 22:56
@vyasr vyasr added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Sep 25, 2023
wence-
wence- requested changes Sep 27, 2023
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor comments with one issue to do with memory-resource management going back and forth between py- and libcudf

python/cudf/cudf/_lib/pylibcudf/interop.pxd Outdated Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/__init__.pxd Outdated Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/interop.pyx Outdated Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/scalar.pyx Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/scalar.pyx Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/scalar.pyx Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/scalar.pyx Show resolved Hide resolved
python/cudf/cudf/_lib/scalar.pyx Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/scalar.pyx Outdated Show resolved Hide resolved
python/cudf/cudf/_lib/pylibcudf/scalar.pyx Outdated Show resolved Hide resolved
@vyasr vyasr changed the base branch from branch-23.10 to branch-23.12 September 28, 2023 00:38
@vyasr
Copy link
Contributor Author

vyasr commented Sep 28, 2023

Retargeting to 23.12.

@vyasr vyasr requested review from wence- and shwina September 28, 2023 00:39
@wence- wence- mentioned this pull request Sep 28, 2023
Open
This was referenced Oct 1, 2023
Closed
Closed
wence-
wence- approved these changes Oct 4, 2023
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Good to go from my side. I think we can just open a doc issue for the outstanding comment about the @no_gc_clear decorator.

@vyasr vyasr mentioned this pull request Oct 4, 2023
Open
@vyasr
Copy link
Contributor Author

vyasr commented Oct 6, 2023

/merge

@rapids-bot rapids-bot bot merged commit 96664ec into rapidsai:branch-23.12 Oct 6, 2023
63 checks passed
@vyasr vyasr deleted the feat/add_pylibcudf_scalar branch October 6, 2023 22:09
@vyasr vyasr mentioned this pull request Feb 27, 2024
Open
@vyasr vyasr added the pylibcudf Issues specific to the pylibcudf package label May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wence- wence- wence- approved these changes

@isVoid isVoid Awaiting requested review from isVoid isVoid is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@shwina shwina Awaiting requested review from shwina

Assignees

@vyasr vyasr

Labels
3 - Ready for Review Ready for review by team CMake CMake build issue feature request New feature or request non-breaking Non-breaking change pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API.
Projects
cuDF Python
Status: Done
cuDF/Dask/Numba/UCX
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vyasr @wence- @shwina