You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
There is currently no way to run cuDF operations in a stream-ordered manner. Since cuDF is deeply tied to the pandas API, there are also limits to how much stream-ordering may be exposed in the public API. pylibcudf has no such restrictions and should allow complete control over streams.
Describe the solution you'd like
libcudf APIs are being incrementally modified to support stream-ordering. Once #13744 is complete, pylibcudf should expose the same functionality in its APIs.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context #13087 and #13509 are examples of where stream-ordered Python APIs could be useful.
The text was updated successfully, but these errors were encountered:
Thanks for adding stream support in libcudf. I came across this while exploring Parquet reading in libcudf and have also submitted a request to expose stream support in Python for a similar performance boost.
FEA:
Parquet reading in libcudf now supports CUDA streams, but this functionality is not yet exposed in Python via cudf or pylibcudf. This omission limits potential performance gains for Python users. Adding multi-stream support for Parquet reading in Python would greatly benefit users by improving I/O and computation pipelining, enabling better GPU utilization.
With stream support now available across all libcudf public APIs, it would be valuable to bring this feature to Python, allowing users to take full advantage of optimized Parquet reading within cudf.
Is your feature request related to a problem? Please describe.
There is currently no way to run cuDF operations in a stream-ordered manner. Since cuDF is deeply tied to the pandas API, there are also limits to how much stream-ordering may be exposed in the public API. pylibcudf has no such restrictions and should allow complete control over streams.
Describe the solution you'd like
libcudf APIs are being incrementally modified to support stream-ordering. Once #13744 is complete, pylibcudf should expose the same functionality in its APIs.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
#13087 and #13509 are examples of where stream-ordered Python APIs could be useful.
The text was updated successfully, but these errors were encountered: