Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StridedMemoryView should expose access to the producer stream if possible #259

Closed
leofang opened this issue Dec 2, 2024 · 1 comment
Closed
Assignees
Labels
cuda.core Everything related to the cuda.core module enhancement Any code-related improvements triage Needs the team's attention

Comments

@leofang
Copy link
Member

leofang commented Dec 2, 2024

In an offline discussion it was brought to my attention that some use cases need a way to order the producer stream after the consumer one, when the work is done. (Ex: a ping pong exchange between producer/consumer in a tight loop.)

Right now, StridedMemoryView follows both DLPack and CAI and does the hand-shake (stream ordering) only once. But a second ordering can be done if the producer stream is exposed. In general, this is not considered safe (because the producer may not guarantee the stream pointer is always valid during the lifetime of a StridedMemoryView instance). However, we do have one situation that guarantees the pointer lifetime, which is when StridedMemoryView instances are created inside a @args_viewable_as_strided_memory context (WIP in #247). In this case, before the decorated function returns the producer stream is considered always valid, and by exposing it we allow users to perform such ordering before this function returns, e.g.

@args_viewable_as_strided_memory((0,))
def my_func(arr, work_stream):
    # work_stream is a "consumer" stream
    view = arr.view(work_stream.handle)
    assert isinstance(view.producer_stream, Stream)
    # ... do work on work_stream ...
    view.producer_stream.wait(work_stream)
    return
@leofang leofang added cuda.core Everything related to the cuda.core module enhancement Any code-related improvements P0 High priority - Must do! labels Dec 2, 2024
@leofang leofang added this to the cuda.core beta 2 milestone Dec 2, 2024
@leofang leofang self-assigned this Dec 2, 2024
@leofang leofang added triage Needs the team's attention and removed P0 High priority - Must do! labels Dec 2, 2024
@leofang leofang removed this from the cuda.core beta 2 milestone Dec 2, 2024
@leofang
Copy link
Member Author

leofang commented Dec 2, 2024

This is not possible with DLPack as we do not have access to the producer stream.

@leofang leofang closed this as not planned Won't fix, can't repro, duplicate, stale Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuda.core Everything related to the cuda.core module enhancement Any code-related improvements triage Needs the team's attention
Projects
None yet
Development

No branches or pull requests

1 participant