Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix race condition with Nanny.on_exit_sync callback #6664

Merged
merged 1 commit into from
Jul 4, 2022

Conversation

hendrikmakait
Copy link
Member

Partially addresses #6625
We have a race condition with the Nanny.on_exit_sync callback, which might be executed after the _ongoing_background_tasks are already closed, resulting in an AsyncTaskGroupClosedError. In that case, something else is already closing the nanny, so we do not need to bother. Ideally, the on_exit and close logic would be generally cleaned up and avoid concurrent invocations, but that will likely need to be a more elaborate fix.

https://github.com/fjetter/distributed/runs/7090337573?check_suite_focus=true#step:11:3425

  • Tests added / passed
  • Passes pre-commit run --all-files

@github-actions
Copy link
Contributor

github-actions bot commented Jul 1, 2022

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       15 files  ±0         15 suites  ±0   6h 43m 33s ⏱️ + 5m 40s
  2 908 tests ±0    2 823 ✔️ +3    81 💤  - 1  3  - 1  1 🔥  - 1 
21 533 runs   - 1  20 576 ✔️  - 1  952 💤 +2  4  - 1  1 🔥  - 1 

For more details on these failures and errors, see this check.

Results for commit 477e76c. ± Comparison against base commit 0f77c0d.

@quasiben
Copy link
Member

quasiben commented Jul 1, 2022

The gpuCI failures are a known issue due problems with rmm/cuda-python. Feel free to merge in

@fjetter fjetter merged commit c992f80 into dask:main Jul 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants