Fix leak in test_localcluster_get_client

[hendrikmakait]

Fixes issue mentioned in #6745 (comment).
Likely fixes:

• Flaky distributed.deploy.tests.test_spec_cluster::test_adaptive_killed_worker #6794
• Flaky distributed.deploy.tests.test_spec_cluster::test_restart #6795
• Flaky tests: assert False, (bad_thread, call_stacks) - Worker executor thread still running after Nanny.kill #6796

which started occurring once #6745 was merged.

• Tests added / passed
• Passes pre-commit run --all-files

[hendrikmakait]

Note: We may want to add an autouse fixture that checks all tests for leaks (#6498).

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       15 files  ±0         15 suites  ±0   6h 49m 29s ⏱️ + 22m 11s
  2 983 tests ±0    2 890 ✔️ +1       88 💤 +1  3 ❌  - 2  2 🔥 ±0 
22 114 runs  ±0  21 077 ✔️ +2  1 031 💤 +3  4 ❌  - 5  2 🔥 ±0 

For more details on these failures and errors, see this check.

Results for commit 4489298. ± Comparison against base commit 9267a21.

[gjoseph92]

This looks like a good change to me. But I'm not sure how it would affect other tests besides test_localcluster_get_client? Are you saying it would address the RuntimeError: cannot schedule new futures after shutdown issue from #6745 (comment)? Because I'm not sure how it would affect #6796 (but I may be missing something).

[gjoseph92]

Going to merge this now regardless since test_localcluster_get_client is causing failures.

[hendrikmakait]

But I'm not sure how it would affect other tests besides test_localcluster_get_client?

To be honest, my assumption was based on correlation rather than an understanding of causality. What I observed was an oddly behaving test that kept the test suite busy for a while after passing and then returned a pretty long error output as well as a sudden uptick in CI failures once this test had been merged into main. Once I had fixed this test, those failures seemed to have disappeared on my branch. Should this really fix the linked CI issues, this might warrant a more thorough investigation of what was going on that this test affected seemingly unrelated ones.