Daphne test server crashes after test of async view

[medihack]

In a Django project, I use pytest-playwright for acceptance tests using Daphne and Django Channels. This works great, but when a test runs that tests an async view (using django.test.Client) beforehand, Daphne crashes with the below error message. So it seems to be an asyncio problem, but I am unsure if the cause is Daphne or Django Channels (or even pytest-playwright). If I skip the async view tests, the following acceptance tests with Daphne work fine.

This is how I set Daphne up for testing: https://github.com/openradx/adit-radis-shared/blob/c0615600a8bab511aa0ce22baa3e6ba11be0ad8d/adit_radis_shared/common/utils/testing_helpers.py#L21
I also tried it with ChannelsLiveServerTestCase and, unfortunately, I got the same error (but still using pytest as a test runner).

Do you have any suggestions on what might be going wrong or how I can investigate this problem further?

[2024-12-25 23:42:14 UTC] daphne.server ERROR Exception inside application: CurrentThreadExecutor already quit or is broken
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/asyncio/tasks.py", line 314, in __step_run_and_handle_result
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/django/contrib/staticfiles/handlers.py", line 101, in __call__
    return await self.application(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/routing.py", line 48, in __call__
    return await application(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/security/websocket.py", line 37, in __call__
    return await self.application(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/sessions.py", line 44, in __call__
    return await self.inner(dict(scope, cookies=cookies), receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/sessions.py", line 261, in __call__
    return await self.inner(wrapper.scope, receive, wrapper.send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/sessions.py", line 44, in __call__
    return await self.inner(dict(scope, cookies=cookies), receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/sessions.py", line 261, in __call__
    return await self.inner(wrapper.scope, receive, wrapper.send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/auth.py", line 185, in __call__
    return await super().__call__(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/middleware.py", line 24, in __call__
    return await self.inner(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/routing.py", line 118, in __call__
    return await application(
           ^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/consumer.py", line 95, in app
    return await consumer(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/consumer.py", line 62, in __call__
    await await_many_dispatch([receive], self.dispatch)
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/utils.py", line 50, in await_many_dispatch
    await dispatch(result)
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/consumer.py", line 73, in dispatch
    await aclose_old_connections()
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/channels/db.py", line 23, in aclose_old_connections
    return await sync_to_async(close_old_connections)()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/asgiref/sync.py", line 455, in __call__
    exec_coro = loop.run_in_executor(
                ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 862, in run_in_executor
    executor.submit(func, *args), loop=self)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/asgiref/current_thread_executor.py", line 115, in submit
    return self._submit(fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pysetup/.venv/lib/python3.12/site-packages/asgiref/current_thread_executor.py", line 99, in _submit
    raise RuntimeError("CurrentThreadExecutor already quit or is broken")
RuntimeError: CurrentThreadExecutor already quit or is broken

[carltongibson]

Hi @medihack — We see a few of these issues for folks using pytest, but no-one has dug in to see what's actually happening (or what the correct workaround would be). It seems to be something to do with the way pytest manages event loops and such on a per-test basis. I'm afraid I can't say more as I'm not a big pytest user.

[medihack]

Thanks for the reply, @carltongibson. Maybe I will switch back to unittest. I am not much of a fan of those pytest fixtures anyway (no IDE autocomplete support in VSC, and they are very obscure sometimes). I'll close the discussion for now and will report back if the problem remains with unittest.