Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid using iterable coroutines in asyncio internally #100112

Closed
kumaraditya303 opened this issue Dec 8, 2022 · 4 comments
Closed

Avoid using iterable coroutines in asyncio internally #100112

kumaraditya303 opened this issue Dec 8, 2022 · 4 comments
Assignees
Labels
3.12 bugs and security fixes topic-asyncio

Comments

@kumaraditya303
Copy link
Contributor

kumaraditya303 commented Dec 8, 2022

asyncio currently uses iterable coroutine directly to wrap awaitables with __await__ methods. This leads to unnecessary special casing and is confusing as asyncio does not supports using yield from now so it should not be used internally too. This will avoid checking for generators everywhere in public APIs (TBD for different issue).

Linked PRs

@kumaraditya303
Copy link
Contributor Author

PR #100128 adds a coroutine wrapper around it so that it is not used directly.

@kumaraditya303 kumaraditya303 self-assigned this Dec 15, 2022
@kumaraditya303
Copy link
Contributor Author

More background on this:

When a custom awaitable object is wrapped by a task, it creates a new iterable coroutine with a generator object as wrapper for the awaitable. This breaks introspection tools which expect a coroutine as they instead get a generator object which is surprising unless you know how things work internally.

import asyncio
import inspect

class Awaitable:
    def __await__(self):
        return asyncio.sleep(1).__await__()

async def main():
    task = asyncio.ensure_future(Awaitable())
    coro = task.get_coro()
    print(f"Task created with {coro=}")
    print(inspect.getcoroutinestate(coro))

asyncio.run(main())

This also somewhat breaks task.get_coro which is expected to return a coroutine object. https://docs.python.org/3/library/asyncio-task.html?highlight=get_coro#asyncio.Task.get_coro

@kumaraditya303
Copy link
Contributor Author

@gvanrossum #100128 fixes the above and always returns coroutine. The PR has conflicts but this is the idea behind it.

@kumaraditya303
Copy link
Contributor Author

This will also help in allowing generators to accepted in more places like wait etc but that's for different issue.

carljm added a commit to carljm/cpython that referenced this issue Mar 17, 2023
* main: (34 commits)
  pythongh-102701: Fix overflow in dictobject.c (pythonGH-102750)
  pythonGH-78530: add support for generators in `asyncio.wait` (python#102761)
  Increase stack reserve size for Windows debug builds to avoid test crashes (pythonGH-102764)
  pythongh-102755: Add PyErr_DisplayException(exc) (python#102756)
  Fix outdated note about 'int' rounding or truncating (python#102736)
  pythongh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (python#102760)
  pythongh-99726: Improves correctness of stat results for Windows, and uses faster API when available (pythonGH-102149)
  pythongh-102192: remove redundant exception fields from ssl module socket (python#102466)
  pythongh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (python#102743)
  pythongh-102737: Un-ignore ceval.c in the CI globals check (pythongh-102745)
  pythonGH-102748: remove legacy support for generator based coroutines from `asyncio.iscoroutine` (python#102749)
  pythongh-102721: Improve coverage of `_collections_abc._CallableGenericAlias` (python#102722)
  pythonGH-102653: Make recipe docstring show the correct distribution (python#102742)
  Add comments to `{typing,_collections_abc}._type_repr` about each other (python#102752)
  pythongh-102594: PyErr_SetObject adds note to exception raised on normalization error (python#102675)
  pythongh-94440: Fix issue of ProcessPoolExecutor shutdown hanging (python#94468)
  pythonGH-100112:  avoid using iterable coroutines in asyncio internally (python#100128)
  pythongh-102690: Use Edge as fallback in webbrowser instead of IE (python#102691)
  pythongh-102660: Fix Refleaks in import.c (python#102744)
  pythongh-102738: remove from cases generator the code related to register instructions (python#102739)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.12 bugs and security fixes topic-asyncio
Projects
Status: Done
Development

No branches or pull requests

1 participant