Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Report all unsupported operations for a query in cudf-polars #16690

Closed
wence- opened this issue Aug 29, 2024 · 1 comment · Fixed by #16960
Closed

[FEA] Report all unsupported operations for a query in cudf-polars #16690

wence- opened this issue Aug 29, 2024 · 1 comment · Fixed by #16960
Assignees
@Matt711
Labels
cudf.polars Issues specific to cudf.polars feature request New feature or request

Comments

@wence-
Copy link
Contributor

wence- commented Aug 29, 2024

Is your feature request related to a problem? Please describe.

Users of cudf-polars can run in to the situation where their (potentially complicated) query doesn't (yet) run on the GPU because we have not implemented support for all of the polars features they are using.

Right now, our fallback mechanism is to fail fast on the first unsupported feature. This works from a correctness point of view, but has some drawbacks:

  • Submitting new feature requests to get a query to run is an iterative (and potentially long-winded process): the first missing feature is requested, only to find that something else is now not supported
  • Consequently, especially if we get these from external users who do not run on bleeding edge nightlies, they might have to wait multiple releases for their query to work (since they must wait to see "what's next").

Describe the solution you'd like

Instead, we should implement a mechanism whereby we traverse the entire query, collecting all the pieces that are not supported, and then raise an error for fallback, reporting all the (unique) unsupported pieces.

This would allow reports for unsupported queries to indicate all the missing pieces, and help us to prioritise things.
@wence- wence- added cudf.polars Issues specific to cudf.polars feature request New feature or request labels Aug 29, 2024
@Matt711 Matt711 self-assigned this Sep 19, 2024
@brandon-b-miller
Copy link
Contributor

Hey @Matt711 , I did a little preliminary investigating of how to do this, let me know if you want to connect and I can share my notes.

@Matt711 Matt711 mentioned this issue Oct 1, 2024
Merged
3 tasks
@GPUtester GPUtester moved this from Todo to In Progress in cuDF Python Oct 2, 2024
rapids-bot bot pushed a commit that referenced this issue Nov 12, 2024
@Matt711
[FEA] Report all unsupported operations for a query in cudf.polars (#…
043bcbd 
…16960)

Closes #16690. The purpose of this PR is to list all of the unique operations that are unsupported by `cudf.polars` when running a query. 

1. Question: How to traverse the tree to report the error nodes? Should this be done upstream in Polars?
2. Instead of traversing the query afterwards, we should probably catch each unsupported feature as we translate the IR.

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)
  - Lawrence Mitchell (https://github.com/wence-)

URL: #16960
@github-project-automation github-project-automation bot moved this from In Progress to Done in cuDF Python Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Matt711 Matt711

Labels
cudf.polars Issues specific to cudf.polars feature request New feature or request
Projects
cuDF Python
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[FEA] Report all unsupported operations for a query in cudf.polars Matt711/cudf
3 participants
@wence- @Matt711 @brandon-b-miller