`asgiref` backward compatibility

[tony]

Checklist

• The bug is reproducible against the latest release or master.
• There are no similar issues or pull requests to fix it yet.

Describe the bug

Hi, thank you for the project, this is a shot in the dark since it's not a channels bug!

#1100 pinned the asgiref dependency to >=3.4.0 for mypy purposes. It may be worth loosening it if it's only mypy as the restriction requires the user to use asgiref 3.4.0, which triggers a well-known but unsolved channels (a different package)

• ASGIRef 3.4.1 + Channels 3.0.3 causes non-deterministic 500 errors serving static files django/channels#1722
• https://stackoverflow.com/q/68208931
• https://stackoverflow.com/q/68343003

Alternatives to pulling in asgiref for typings:

1. Dropping the asgiref dependency and copying asgiref.typings to _types

   _types can have compat-like functionality depending on the asgiref version installed so things so uvicorn can be independent of django and channels.

2. Loosening the constraint

   Would mypy work if try/catch with ImportError was used? Would be willing to consider it as a compatibility workaround until ASGIRef 3.4.1 + Channels 3.0.3 causes non-deterministic 500 errors serving static files django/channels#1722 is resolved?

   try:
       from asgiref.typing import WebSocketScope
   except ImportError:
      from asgiref.typing import WebsocketScope
      WebSocketScope = WebsocketScope

A word of note:

This is for posterity, of course it's never needed for an independent project such as uvicorn to change it's pinning due to another package's issues, but noteworthy since channels is often used alongside uvicorn by users.

The reasons why I post this here is channel's bug at django/channels#1722 is prominent, its easiest apparent workaround is to decrease asgiref's version below <3.4.0, and the only reason I can see 3.4.0 was pinned in #1100 is mypy and there may be a way to satisfy mypy and relax the constraint.

I think it's up to the maintainer and I wanted to document the issue.

Steps to reproduce the bug

I believe this happens on sites where static files are served via the asgi server. This happens when there's a heavy load (such as selenium tests).

Expected behavior

Static files via dev servers should serve without raising deadlock errors

Actual behavior

No response

Debugging material

Assuming uvicode 0.16.0, it requires we use asgiref 3.4.0+

Error: 1-03 17:27:52 +0000] [9] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/opt/venv/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 376, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/opt/venv/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "/opt/venv/lib/python3.10/site-packages/uvicorn/middleware/asgi2.py", line 17, in __call__
    await instance(receive, send)
  File "/opt/venv/lib/python3.10/site-packages/channels/http.py", line 192, in __call__
    await self.handle(body_stream)
  File "/opt/venv/lib/python3.10/site-packages/asgiref/sync.py", line 409, in __call__
    raise RuntimeError(
RuntimeError: Single thread executor already being used, would deadlock

Environment

OS: Ubuntu latest via docker
uvicorn: 0.16.0
channels: 2.4.0
asgiref: 3.4.0 (breakage) 3.3.4 (works)
python 3.10

Additional context

No response

[tony]

If it were me, I'd vendorize the typings again and check the asgiref.__version__ to handle at compatibility differences.

The reason why is asgiref 3.3 and 3.4, despite being minor version bumps and technically safer, are fundamentally different when placed into a server or CI environment. If you're using channels you've opened up a can of worms of static file bugs when channels and django comes into play.

Perhaps future asgiref releases won't be as drastic as 3.3 and 3.4, but I think that typings alone aren't enough of a reason to include the package in uvicorn and add a constraint. asgiref is not stable enough yet when other packages like channels, django and normal CI/devops is concerned.

While I see the ecosystem around it stabilizing over time, I am happy to open a PR to re-vendorize typings in and with multi-version asgiref compatibility (and tests).

[Kludex]

I'm willing to approve a PR with the vendor, and tests running against asgiref.typing. 👍

[tony]

@Kludex I will prepare one ASAP

[Kludex]

Cool. Thanks 👍

[tony]

Follow up in #1312

_types can have compat-like functionality depending on the asgiref version installed so things so uvicorn can be independent of django and channels.

That hypothesis isn't working out. I don't see a way to mypy or type annotations can branch based on asgiref version.

[tomchristie]

This is me buzzing by with the observation that just mellowing out and rolling with scope: dict is cool and relaxed and fuzzywuzzy, and makes for more friendly codebases. (At least from my perspective) 😌

[tony]

This is me buzzing by with the observation that just mellowing out and rolling with scope: dict is cool and relaxed and fuzzywuzzy, and makes for more friendly codebases. (At least from my perspective) 😌

opened at #1373

[tony]

@tomchristie Would you want to move this to discussion per #1374 until there's a consensus? Or fine keeping it as issue for now?

[tomchristie]

I'm not sure @tony - that's a good question.

I'd certainly suggest closing #1373 off, yes.

Since this issue is open, and since it's clear that it's behaviourally breaking something for you let's keep it open and get it resolved.

Your option (2.) seems like it might be a good approach to start with.
That'd be a nice low-footprint change, which I'm all in favour of.

[tony]

I will keep this issue open for now

I'd certainly suggest closing #1373 off, yes.

Can you clarify what's meant by closing it off? #1305 (comment) discussed using dict in favor of Scope.

[tomchristie]

Can you clarify what's meant by closing it off?

Yes. I think we should close #1373. (Without merging.)

#1305 (comment) discussed using dict in favor of Scope.

I think there's a discussion worth having there, but not we should definitely do this.

I will keep this issue open for now.

Agreed.

The lowest-possible change footprint PR is probably a good approach. If (2) would resolve the issue for you then I'd suggest we'd want to go with that.

We could later consider either vendoring asgiref as per #1312, or dropping it by relaxing the typing, but those options can start as a discussion. (And there's no rush on them if they're not behaviourally breaking things)