[core] Fix deadlock when cancelling stale requests on in-order actors

[dayshah]

Problem

Today we hold stop_mu while calling into ActorSchedulingQueue::Add. This calls into ActorSchedulingQueue::ScheduleRequests which can potentially cancel and therefore call the cancel callback which also tries to acquire stop_mu in the same call stack.

Cancel inside ActorSchedulingQueue::ScheduleRequests

ray/src/ray/core_worker/task_execution/actor_scheduling_queue.cc

Line 174 in c6b8c9f

head->second.Cancel(

cancel_callback grabbing stop_mu

ray/src/ray/core_worker/task_execution/task_receiver.cc

Line 176 in c6b8c9f

absl::MutexLock lock(&stop_mu_);

Grabbing stop_mu in TaskReceiver::HandleTask which eventually leads into ActorSchedulingQueue::ScheduleRequests

ray/src/ray/core_worker/task_execution/task_receiver.cc

Line 195 in c6b8c9f

absl::MutexLock lock(&stop_mu_);

Solution

The solution here is just to turn stopping_ into an atomic bool. The mutex only exists to protect this.

Extra

• Renaming test_transient_error_retry to test_push_actor_task_failure and moving it and test_update_object_location_batch_failure to test_core_worker_fault_tolerance
• out_of_order=False case for the push task failure test

[gemini-code-assist]

Code Review

This pull request effectively resolves a deadlock that occurred when canceling stale requests for in-order actors. The approach of replacing the mutex-guarded boolean stopping_ with a std::atomic<bool> is a clean and correct solution to the re-entrant lock problem. The changes in task_receiver.cc and task_receiver.h are well-implemented. Additionally, moving and extending the Python tests to cover the in-order execution case is a valuable addition that ensures the fix is properly verified. Overall, this is a solid improvement to the codebase's stability. I have one minor suggestion for code simplification.