Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nest_asyncio is incompatible with uvloop #2941

Open
ff137 opened this issue May 9, 2024 · 0 comments
Open

nest_asyncio is incompatible with uvloop #2941

ff137 opened this issue May 9, 2024 · 0 comments

Comments

@ff137
Copy link
Contributor

ff137 commented May 9, 2024

nest_asyncio is used in the JSON-LD document loader methods (aries_cloudagent/vc/ld_proofs/document_loader.py)

The incompatibility was observed when working with the redis_events plugin in hyperledger/aries-acapy-plugins, which uses uvicorn. After attempting to update uvicorn from 0.16 to 0.29 over there, it appears that one of the changes introduces uvloop, causing an incompatibility with the nest_asyncio framework.
The error stacktrace is as follows:

2024-05-09 14:48:58,232 aries_cloudagent.core.dispatcher ERROR Handler error: validation_middleware
Traceback (most recent call last):
  File "/home/aries/.local/lib/python3.9/site-packages/pyld/context_resolver.py", line 143, in _fetch_context
    remote_doc = jsonld.load_document(url,
  File "/home/aries/.local/lib/python3.9/site-packages/pyld/jsonld.py", line 6583, in load_document

=================
    remote_doc = options['documentLoader'](url, options)
  File "/home/aries/.local/lib/python3.9/site-packages/aries_cloudagent/vc/ld_proofs/document_loader.py", line 111, in __call__
    document = loop.run_until_complete(coroutine)
  File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete
  File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever
  File "uvloop/loop.pyx", line 518, in uvloop.loop.Loop._run
RuntimeError: this event loop is already running.

Looking at the commit history, I can see that nest_asyncio was introduced as a workaround:
40e40eb

fix: Replace hack within document loader

After talking with Timo, the document loader uses a seperate thread due
to the fact that PyLD is synchronous code and calls into the document
loader (which requires asynchronous code). When swapping out the cache
with a Redis based cache, new exceptions rose up due to this
implementation. No matter what I tried, it seemed impossible to keep the
separate thread/asyncio event loop due to the error `got Future attached
to a different loop`.

Switching to `nest_asyncio` over using a separate thread/event loop
resolves the issues that I was observing.

Signed-off-by: Colton Wolkins (Indicio work address) <colton@indicio.tech>

I'm logging this as an issue so that we can find an alternative to nest_asyncio, which is compatible with uvloop.

A refactoring of asyncio event loops may in any case be desirable because "uvloop makes asyncio 2-4x faster": https://github.com/MagicStack/uvloop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant