Eager Asynchronous Construction of App Interferes with Running Event Loop

[MarioIshac]

Because Server#run calls Config#load within loop.run_until_complete(self.serve(...)), a config load that involves asynchronously constructing an app fails because the event loop is already running. Note that this config load will fail only if the app name is provided instead of the app itself. This is because in the latter case, the app had already been asynchronously constructed in its own run_until_complete prior to the run_until_complete launched by Server.

To reproduce

1. pip install -e git+https://github.com/encode/uvicorn#egg=uvicorn
2. Create main.py:

import asyncio

async def new_app():
    async def app(scope, receive, send):
        await send({
            'type': 'http.response.start',
            'status': 200,
            'headers': [
                [b'content-type', b'text/plain'],
            ],
        })
        await send({
            'type': 'http.response.body',
            'body': b'Hello, world!',
        })

    return app

app = asyncio.get_event_loop().run_until_complete(new_app())

3. Run uvicorn main:app.

Expected behavior

INFO: Started server process [xxx]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

And that a server request would return Hello, world!

Actual behavior

Traceback (most recent call last):
  File "/home/mishac/dev/python/personal/async_init/venv/bin/uvicorn", line 33, in <module>
    sys.exit(load_entry_point('uvicorn', 'console_scripts', 'uvicorn')())
  File "/home/mishac/dev/python/personal/async_init/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/mishac/dev/python/personal/async_init/venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/mishac/dev/python/personal/async_init/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mishac/dev/python/personal/async_init/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/mishac/dev/python/personal/async_init/venv/src/uvicorn/uvicorn/main.py", line 362, in main
    run(**kwargs)
  File "/home/mishac/dev/python/personal/async_init/venv/src/uvicorn/uvicorn/main.py", line 386, in run
    server.run()
  File "/home/mishac/dev/python/personal/async_init/venv/src/uvicorn/uvicorn/server.py", line 49, in run
    loop.run_until_complete(self.serve(sockets=sockets))
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/home/mishac/dev/python/personal/async_init/venv/src/uvicorn/uvicorn/server.py", line 56, in serve
    config.load()
  File "/home/mishac/dev/python/personal/async_init/venv/src/uvicorn/uvicorn/config.py", line 308, in load
    self.loaded_app = import_from_string(self.app)
  File "/home/mishac/dev/python/personal/async_init/venv/src/uvicorn/uvicorn/importer.py", line 20, in import_from_string
    module = importlib.import_module(module_str)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "./main.py", line 19, in <module>
    app = asyncio.get_event_loop().run_until_complete(new_app())
  File "uvloop/loop.pyx", line 1450, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1443, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1351, in uvloop.loop.Loop.run_forever
  File "uvloop/loop.pyx", line 480, in uvloop.loop.Loop._run
RuntimeError: this event loop is already running.

Event loop started running before app could be asynchronously constructed.

Environment

Ubuntu 20.04
CPython 3.8.5
Uvicorn 0.13.3

Additional context

Replacing step 1 with pip install -e git+https://github.com/MarioIshac/uvicorn@feat-async-construction#egg=uvicorn will achieve the expected behavior.

The particular benefit I get from having the app asynchronously & eagerly constructed instead of lazily constructed within async def app(...) is that I can take advantage of gunicorn preloading. I can also have the server fail-fast instead of on-first-request in the case that the asynchronous construction fails (for example, database connection pool could not be created).

Important

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[florimondmanca]

@MarioIshac Hello,

To be honest, this is looking like a very odd setup to me.

Have you considered leveraging the ASGI lifespan functionality? Apps can receive "startup" and "shutdown" events. If startup event handlers fail then Uvicorn will fast exit, just as the use case you described.

I believe this would be a much saner approach than a "create app a synchronously".

Other than that, to resolve the "bug" you're seeing, you'd have to properly close the event loop once your app has been created. Right now you're leaving it hanging unclosed, but Uvicorn expects a clean slate async environment.

[florimondmanca]

@MarioIshac Ill close this preemptively since I think you should be able to figure things out given my hints above. Don't hesitate to reach back if need be. Thanks!

[MarioIshac]

Hi @florimondmanca,

Thank you for your reply. I'll admit I only recently jumped into asyncio, so it was hasty of me to classify this as a "bug" instead of asking a question first.

I actually do use ASGI lifespan functionality to handle initialization that needs to be done per-process (because of library incompatibility with gunicorns --preload), while I wanted to use asyncio.get_event_loop().run_until_complete to handle initialization that could be done in the parent process. For this example:

import asyncio

async def new_app():
    print("Pre-fork initialization.")
    async def app(scope, receive, send):
        if scope['type'] == 'lifespan':
            while True:
                message = await receive()
                if message['type'] == 'lifespan.startup':
                    print("Post-fork initialization.")
                    await send({'type': 'lifespan.startup.complete'})
    return app

app = asyncio.get_event_loop().run_until_complete(new_app())

On the branch I made, running with gunicorn main:app -k uvicorn.workers.UvicornWorker -w 2 prints Pre-fork initialization and Post-fork initialization twice as expected, but running with --preload prints the former only once.

So while the ASGI startup event functionality does come close (it is still eager), I haven't found a way to make it benefit from preloading.

Separate from this, I have a question about the last point you made. Since the app is loaded from a string, I thought uvicorn would be still creating the event loop (and selecting uvloop / asyncio appropriately), with the only thing changing being that the run_until_complete for app initialization is done outside of the run_until_complete for serving requests, instead of within. In my mind, I changed this:

async def a():  # my code
    pass

async def b():  # uvicorn code
    event_loop.run_until_complete(a())

event_loop = asyncio.new_event_loop()  # uvicorn code
event_loop.run_until_complete(b())
event_loop.close()  # uvicorn code

to this:

async def a(): 
    pass

async def b(): 
    pass

event_loop = asyncio.new_event_loop()
event_loop.run_until_complete(a())
event_loop.run_until_complete(b())
event_loop.close()

So asyncio.get_event_loop() in my example shouldn't be creating a new event loop, only using the one uvicorn created (but I think this assumption breaks down if the app is provided by object instead of by name). Could you clarify what parts of my mental model here is wrong?

[florimondmanca]

@MarioIshac When you run uvicorn main:app, Uvicorn runs the uvicorn.main() CLI function, which calls uvicorn.run(). If you look in our code you'll see that this basically does:

config = Config(...)
server = Server(config)
server.run()

The event loop is set up by server.run(), which calls something like config.setup_loop(). It then calls await server.serve() within the event loop, which loads the app and runs the main server loop. So your app module actually gets loaded within the context of a running event loop. So effectively what happens is:

def appmodule():  # your code
    loop = asyncio.get_event_loop()
    return loop.run_until_complete(await getapp())  # XXX: Can't do this: `loop` is already running!

async def serve():  # uvicorn code
    app = appmodule().app
    await _mainloop(app)

# uvicorn code
loop = asyncio.new_event_loop()
loop.run_until_complete(serve())
loop.close()

[MarioIshac]

@florimondmanca Yup I looked over this section of Server, what my merge request did was change it into effectively:

# uvicorn code
loop = asyncio.new_event_loop()
app = loop.run_until_complete(getapp())

async def serve():  # uvicorn code
    await _mainloop(app)

loop.run_until_complete(serve())
loop.close()

What is the downside of such an approach? As far as I understand uvicorn is still controlling the event loop. Benefit of this approach is now I can advantage of preloading as shown in earlier reply. Whereas for things I want to control shutdown for and/or initialize per-process, I'd use ASGI lifespan.

[florimondmanca]

@MarioIshac Okay — my bad, I didn't actually see your PR #942. :-) That's interesting, let me take a look.

[graingert]

asyncio.get_event_loop().run_until_complete(new_app()) is not supported in python3.10+

[graingert]

I think this would be fixed by #1151

[telemmaite]

I have a similar issue, when trying to use the gunicorn --preload when my preload part/function is async based, I am loading few GBs ML models in the parent process hoping to be used by all forked workers.
Error is Event loop is already closed this shows 1 time per forked child process.

How should I fix this? Do I need two different loops to happen on parent process level or each forked worker needs a fresh own loop ?

[graingert]

https://bugs.python.org/issue21998 asyncio can't survive a fork, you have to start a new loop and make sure the old loop has finished before forking - or use spawn

[caeus]

+1

[bhelabhav]

+1