Import of Dask prevents async subprocess management on Windows

[zanieb]

Dask changes the event loop policy to the Python 3.7 default instead of the Python 3.8+ default. This event loop does not support spawning subprocesses, which means that if you're writing an application that manages async processes and you've imported Dask, your application will fail with a NotImplementedError from within asyncio.

distributed/distributed/config.py

Line 189 in fae59c4

asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

It looks like this was done because Tornado 6 did not support the new event loop policy, but at tornadoweb/tornado#2608 (comment) they note that this should be fixed in Tornado 6.1.

I see an existing issue at #7434 that notes this breaks a specific Dask cluster type, but more importantly this means that applications that import Dask will not be able to manage subprocesses. This is causing problems over at Prefect since our users import Dask and we need to manage subprocesses within our own library :) we're tracking this at PrefectHQ/prefect#8206 and you can see a reproduction there.

I don't have access to a Windows machine myself right now and am coordinating to produce a MRE that doesn't use Prefect code. However, it should look roughly like:

import asyncio
import dask.distributed

asyncio.run(asyncio.create_subprocess_shell("echo hello world"))

The traceback is also roughly like:

Traceback (most recent call last):
  ...
    process = await asyncio.create_subprocess_shell(
  File "C:\Python310\lib\asyncio\subprocess.py", line 205, in create_subprocess_shell
    transport, protocol = await loop.subprocess_shell(
  File "C:\Python310\lib\asyncio\base_events.py", line 1634, in subprocess_shell
    transport = await self._make_subprocess_transport(
  File "C:\Python310\lib\asyncio\base_events.py", line 498, in _make_subprocess_transport
    raise NotImplementedError
NotImplementedError

[fjetter]

There was an attempt to remove this over #5833 but we never followed up. The GH CI logs are unfortunately not available anymore.

[fjetter]

I opened #7494 with all the necessary changes for us to switch. Let's see what CI says

[hendrikmakait]

xref: #7434

[zanieb]

We've added PrefectHQ/prefect#8184 as a workaround for our CLI entrypoints to alleviate the most pressing issues for now. Thanks for hoping on this!