-
-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a warning when setting up async fixtures #10839
Add a warning when setting up async fixtures #10839
Comments
This approach may collide with anyio which uses the standard https://anyio.readthedocs.io/en/stable/testing.html#asynchronous-fixtures |
Okay, so there's several things going on here:
|
great summary @jakkdl , long story short I introduced pytest-random-order to a pretty large suite, it has only one autouse async fixture, and as soon as you get a sync test executed first, then the autouse is silently ignored, and it was a real pain to figure out why. see https://github.com/euri10/pytest_autouse for a reproduction a warning of some sort would be highly appreciated 💌 |
EDIT: I ultimately came up with a clean solution in #12930, this comment can be disregarded Okay "but this is always a user error, so pytest can raise an error without any problem - and whether that happens before or after a plugin does so doesn't matter" was an overstatement, after playing around in #12930 I couldn't come up with an easy way to have pytest error without breaking any plugins or end users code. There are plugins (e.g. hypothesis) that wrap async tests to look sync to pytest, but still handles async fixtures; and end users can write @pytest.fixture
async def fix():
return 1
def test_fix(fix):
assert 1 == asyncio.run(fix) but I also don't love punting this off to all the various plugins to handle, in large part since you can encounter this without having any plugins installed. The only reasonable solution I came up with in #12930 is to require a But perhaps cleaner would be to robustly resolve #10404, although that can still confuse users due to caching import pytest
@pytest.fixture(scope='session')
async def my_fixture():
return 5
@pytest.mark.anyio
async def test_foo_async(my_fixture):
assert my_fixture == 5
# only works if test_foo_async is run before it, and raises no warnings
def test_foo(my_fixture):
assert my_fixture == 5 |
What's the problem this feature will solve?
Pytest cannot evaluate fixture that are async functions or async generators. Users of async pytest plugins may unintentionally annotate those functions with
@pytest.fixture
rather than the async plugin's fixture function.The fixture result ends up to be an unawaited async generator or coroutine, which is not what the user expected. Most of the time, the corresponding tests will fail. However, when the user defines an autouse fixture to perform setup or teardown, the fixture can silently fail.
Fixture results in async generator
Fixture results in coroutine
Fixture silently does nothing
Describe the solution you'd like
Pytest emits the following warning when running async test functions without an async plugin (see #2224):
The warning is emitted as part of a trylast hook to
pytest_pyfunc_call
. That means async pytest plugins can create hook wrappers to synchronize the test function and prevent the warning from being emitted.I suggest to add the same behavior to the use of
@pytest.fixture
on async functions or async generators.Examples
autouse=True
not executing properly in test pytest-asyncio#518Alternative Solutions
#9962 proposes a mechanism to await results of awaitable tests and fixtures. It can potentially address the issue described here.
Additional context
From the above list of async plugins, only anyio, pytest-asyncio, and pytest-trio make use of pytest_fixture_setup.
The text was updated successfully, but these errors were encountered: