-
Notifications
You must be signed in to change notification settings - Fork 6.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Core] ray.actor.exit_actor() does not seem to work from within an async background thread #49451
Comments
@shomilj Is this bug only occurring occasionally, or can it always be reproduced using your reproduction script? I tried your script but couldn't reproduce it. Update: I can reproduce it now. |
@MortalHappiness I think we can fix this by unifying the asyncio exit path with the ray/src/ray/core_worker/core_worker.cc Line 3165 in fef9596
Basically, when |
@edoakes Thanks for your information! |
…o terminate async actor Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
…o terminate async actor Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
…o terminate async actor Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
Closes: ray-project#49451 Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
When an asyncio task creates another asyncio task, raising `AsyncioActorExit` cannot make the caller exit because they are not the same task. Therefore, this PR makes `exit_actor` to request actor exit in core worker context, which will be checked regularly by core worker. Closes: #49451 --------- Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
…t#49918) When an asyncio task creates another asyncio task, raising `AsyncioActorExit` cannot make the caller exit because they are not the same task. Therefore, this PR makes `exit_actor` to request actor exit in core worker context, which will be checked regularly by core worker. Closes: ray-project#49451 --------- Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
What happened + What you expected to happen
In actor initialization, I want to spawn an async background routine that has the ability to call
ray.actor.exit_actor()
; however, that API doesn't seem to actually work -The actor stays alive (according to Ray dashboard) instead of exiting as I would expect.
Versions / Dependencies
Ray 2.40.0
Reproduction script
Issue Severity
Medium: It is a significant difficulty but I can work around it.
The text was updated successfully, but these errors were encountered: