Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latest 8.0.0 is breaking on 3.8+ #11868

Closed
Joshuaalbert opened this issue Jan 28, 2024 · 22 comments
Closed

Latest 8.0.0 is breaking on 3.8+ #11868

Joshuaalbert opened this issue Jan 28, 2024 · 22 comments

Comments

@Joshuaalbert
Copy link

Joshuaalbert commented Jan 28, 2024

Comment by pytest maintainers: pytest-asyncio is not yet compatible with pytest 8; if you're using pytest-asyncio, stick with pytest<8 until the issue is resolved.


Description

Prior to 8.0.0 all tests on 3.7 -- 3.11 were passing. Then today when 8.0.0 was released 3.8+ started failing.

See screen shot, rerunning tests that passed on 27 Jan, failed today on 28 Jan.
Screenshot from 2024-01-28 13-43-43

Failure is happening on 3.8+
Screenshot from 2024-01-28 13-43-25

The only requirements for the project are pytest and pytest-asyncio. For reference, the project is FairAsyncRLock. The git action workflow for the Ci/Cd is here.

Traceback

This is the problem. It's the same for all 3.8+.

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/main.py", line 272, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/main.py", line 325, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_manager.py", line 119, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 138, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/logging.py", line 783, in pytest_collection
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/warnings.py", line 1[18](https://github.com/Joshuaalbert/FairAsyncRLock/actions/runs/7680017808/job/20940748975#step:6:19), in pytest_collection
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 121, in _multicall
INTERNALERROR>     teardown.throw(exception)  # type: ignore[union-attr]
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/config/__init__.py", line 1365, in pytest_collection
INTERNALERROR>     return (yield)
INTERNALERROR>             ^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/main.py", line 336, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/main.py", line 799, in perform_collect
INTERNALERROR>     self.items.extend(self.genitems(node))
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/main.py", line 942, in genitems
INTERNALERROR>     yield from self.genitems(subnode)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/main.py", line 937, in genitems
INTERNALERROR>     rep, duplicate = self._collect_one_node(node, handle_dupes)
INTERNALERROR>                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/main.py", line 825, in _collect_one_node
INTERNALERROR>     rep = collect_one_node(node)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/_pytest/runner.py", line 566, in collect_one_node
INTERNALERROR>     ihook.pytest_collectstart(collector=collector)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_hooks.py", line 501, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_manager.py", line 1[19](https://github.com/Joshuaalbert/FairAsyncRLock/actions/runs/7680017808/job/20940748975#step:6:20), in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 138, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pluggy/_callers.py", line 102, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/pytest_asyncio/plugin.py", line 6[26](https://github.com/Joshuaalbert/FairAsyncRLock/actions/runs/7680017808/job/20940748975#step:6:27), in pytest_collectstart
INTERNALERROR>     pyobject = collector.obj
INTERNALERROR>                ^^^^^^^^^^^^^
INTERNALERROR> AttributeError: 'Package' object has no attribute 'obj'
@arossert

This comment was marked as duplicate.

@tandav
Copy link

tandav commented Jan 28, 2024

Having the same issue on Python3.12 and pytest-asyncio

@sthagen
Copy link

sthagen commented Jan 28, 2024

When removing pytest-asyncio the error "onion" shows the next problem (version info at the start and deleted all dot cache dirs applicable like .hypothesis and .pytest_cache before the run):

% pytest --version && python --version && pytest -vv test/test_placeholder.py::test_load_svg_placeholder_external
pytest 8.0.0
Python 3.10.13
Test session starts (platform: darwin, Python 3.10.13, pytest 8.0.0, pytest-sugar 0.9.7)
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/some/where/else/.hypothesis/examples'))
metadata: {'Python': '3.10.13', 'Platform': 'macOS-14.2.1-arm64-arm-64bit', 'Packages': {'pytest': '8.0.0', 'pluggy': '1.3.0'}, 'Plugins': {'pytest_check_links': '0.9.0', 'snapshot': '0.9.0', 'harvest': '1.10.4', 'timeout': '2.2.0', 'cases': '3.8.2', 'html': '4.1.1', 'checkdocs': '2.10.1', 'subtests': '0.11.0', 'unordered': '0.5.2', 'steps': '1.8.0', 'hypothesis': '6.97.1', 'todo': '0.2.1', 'cov': '4.1.0', 'shutil': '1.7.0', 'Faker': '22.5.1', 'subprocess': '1.5.0', 'tap': '3.4', 'metadata': '3.0.0', 'reporter': '0.5.2', 'flake8': '1.1.1', 'mock': '3.12.0', 'speed': '0.3.5', 'check': '2.3.1', 'shell': '0.3.2', 'profiling': '1.7.0', 'xdist': '3.5.0', 'sugar': '0.9.7', 'race': '0.2.0', 'anyio': '4.2.0', 'pytest_codeblocks': '0.17.0', 'pydocstyle': '2.3.2', 'threadleak': '0.5.0', 'sherlock': '0.3.4'}}
rootdir: /some/where/else
plugins: pytest_check_links-0.9.0, snapshot-0.9.0, harvest-1.10.4, timeout-2.2.0, cases-3.8.2, html-4.1.1, checkdocs-2.10.1, subtests-0.11.0, unordered-0.5.2, steps-1.8.0, hypothesis-6.97.1, todo-0.2.1, cov-4.1.0, shutil-1.7.0, Faker-22.5.1, subprocess-1.5.0, tap-3.4, metadata-3.0.0, reporter-0.5.2, flake8-1.1.1, mock-3.12.0, speed-0.3.5, check-2.3.1, shell-0.3.2, profiling-1.7.0, xdist-3.5.0, sugar-0.9.7, race-0.2.0, anyio-4.2.0, pytest_codeblocks-0.17.0, pydocstyle-2.3.2, threadleak-0.5.0, sherlock-0.3.4
collecting ...
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ERROR collecting test/test_placeholder.py ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/nodes.py:152: in _create
    return super().__call__(*k, **kw)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/python.py:1801: in __init__
    fixtureinfo = fm.getfixtureinfo(self, self.obj, self.cls)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/fixtures.py:1490: in getfixtureinfo
    names_closure, arg2fixturedefs = self.getfixtureclosure(
E   TypeError: getfixtureclosure() got an unexpected keyword argument 'initialnames'

During handling of the above exception, another exception occurred:
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/pluggy/_hooks.py:493: in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/pluggy/_manager.py:115: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/python.py:277: in pytest_pycollect_makeitem
    return list(collector._genfunctions(name, obj))
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/python.py:486: in _genfunctions
    definition: FunctionDefinition = FunctionDefinition.from_parent(
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/python.py:1809: in from_parent
    return super().from_parent(parent=parent, **kw)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/nodes.py:275: in from_parent
    return cls._create(parent=parent, **kw)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/nodes.py:167: in _create
    return super().__call__(*k, **known_kw)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/python.py:1801: in __init__
    fixtureinfo = fm.getfixtureinfo(self, self.obj, self.cls)
/some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/fixtures.py:1490: in getfixtureinfo
    names_closure, arg2fixturedefs = self.getfixtureclosure(
E   TypeError: getfixtureclosure() got an unexpected keyword argument 'initialnames'
collected 0 items / 1 error

========================================================================================================================== warnings summary ==========================================================================================================================
../../../../../.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/nodes.py:158
  /some/where/.pyenv/versions/3.10.13/envs/liitos-3-10-13/lib/python3.10/site-packages/_pytest/nodes.py:158: PytestDeprecationWarning: <class '_pytest.python.FunctionDefinition'> is not using a cooperative constructor and only takes {'parent', 'name', 'callobj'}.
  See https://docs.pytest.org/en/stable/deprecations.html#constructors-of-custom-pytest-node-subclasses-should-take-kwargs for more details.
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================================================================== short test summary info =======================================================================================================================
FAILED test/test_placeholder.py - TypeError: getfixtureclosure() got an unexpected keyword argument 'initialnames'

Results (0.28s):
ERROR: found no collectors for /some/where/else/test/test_placeholder.py::test_load_svg_placeholder_external

@shayts7

This comment was marked as duplicate.

Joshuaalbert added a commit to Joshuaalbert/FairAsyncRLock that referenced this issue Jan 28, 2024
@bluetech
Copy link
Member

Seems like pytest-asyncio isn't compatible with pytest 8 yet, so until this is fixed, stick with pytest<8. Hopefully pytest-asyncio can do a release which requires pytest<8 soon (until pytest 8 support is added) so this happens automatically.


@sthagen This error indicates that one of the plugins you use is overriding the internal getfixtureclosure function. Because the plugin name does not appear in the stack trace, it's not possible to know which. Perhaps you can do a grep -R 'def getfixtureclosure' in your venv directory to find the plugin (excluding pytest itself of course).

@yaelmi3

This comment was marked as duplicate.

@pablobuenaposada

This comment was marked as duplicate.

@sthagen
Copy link

sthagen commented Jan 28, 2024

@bluetech

Seems like pytest-asyncio isn't compatible with pytest 8 yet, so until this is fixed, stick with pytest<8. Hopefully pytest-asyncio can do a release which requires pytest<8 soon (until pytest 8 support is added) so this happens automatically.

@sthagen This error indicates that one of the plugins you use is overriding the internal getfixtureclosure function. Because the plugin name does not appear in the stack trace, it's not possible to know which. Perhaps you can do a grep -R 'def getfixtureclosure' in your venv directory to find the plugin (excluding pytest itself of course).

Thanks to all the support for changes by providing pytest in the first place to those that do. Having that out of the way, I often follow the dream of semantic versioning and packaging systems until I cannot. Then I take a break and at some point follow our dream again:
I have more than 70 projects using pytest as test dependency, so I simply restricted purest for now in my projects to only use pytest < 8. No problem.

PS: Neither grep nor ripgrep found anything like even only case-insensitive getfixtureclosure in my source tree.

In case you need some feedback for fixed pre-releases, GitHub-commits, anything, please feel free to mention me in such a PR or issue and I will try to provide whatever the maintainers ask for.

PPS: Why does GitHub-WebUI always silently replace my pytest with purest even when I quote and continue writing a space ... 🤷🏽‍♂️

@bluetech
Copy link
Member

The getfixtureclosure crash likely comes from pytest-cases according to smarie/python-pytest-cases#330.

@seifertm
Copy link

Pytest-asyncio specifies an upper bound on the supported pytest version since v0.23.4a2. Unfortunately, the pre-release didn't make it in time for the pytest 8 release.

I just released v0.23.4 which correctly specifies the compatible pytest versions. This should prevent pytest-asyncio from being installed together with pytest 8 by accident, until the compatibility issue is resolved on the pytest-asyncio side.

Sorry for causing noise on the pytest bug tracker. Feel free to ping me directly in this kind of issues.

@nicoddemus
Copy link
Member

Thanks folks, closing this then as there's nothing to be done on pytest's side.

@nicoddemus nicoddemus closed this as not planned Won't fix, can't repro, duplicate, stale Jan 28, 2024
@sscherfke
Copy link
Contributor

This also breaks Sybil: simplistix/sybil#107

@nicoddemus Maybe you could add this API change to the changelog so that people can find information about it and some hints on how to fix it in their code?

@sscherfke
Copy link
Contributor

The change with getfixtureclosure() is that ignore_args: Sequence[str] = () became ignore_args: AbstractSet[str] (w/o a default). Maybe Pytest could change that to ignore_args: AbstractSet[str] = frozenset(). This immutable default might help to keep the API more or less backwards compatible.

@cjw296
Copy link

cjw296 commented Jan 29, 2024

@nicoddemus - any thoughts on stabalising a fixture requesting API? It feels like many pytest releases break this stuff and it's pretty wearing :-(

@The-Compiler
Copy link
Member

#2471 is probably the better place for this.

@cjw296
Copy link

cjw296 commented Jan 29, 2024

@The-Compiler - yes, well, 5 years later and that issue has still seen no progress...

@The-Compiler
Copy link
Member

That doesn't make an issue about an unrelated problem (the original post is something entirely different) a more suitable place.

@nicoddemus
Copy link
Member

nicoddemus commented Jan 30, 2024

@nicoddemus - any thoughts on stabalising a fixture requesting API? It feels like many pytest releases break this stuff and it's pretty wearing :-(

I completely understand the feeling, and it is frustrating to us as well... believe us, we don't make light to introduce breaking changes, even for internal APIs -- @bluetech goes to great pains grepping a large number of plugins to check if internal changes are breaking them, for example.

Having said that, we have been doing some nice refactorings in the direction of cleaning up pytest internals (especially @bluetech and @sadra-barikbin, who have been doing fantastic work). Unfortunately much of the pytest internal design/implementation just grew organically over the years (pytest is really old, started in 2008 or so I believe), without an overall design driving the changes, so we are playing catch up -- and the moving is slow, because we have limited time as maintainers and also because we realize we should make baby steps in order to avoid breaking things; and even so, we end up breaking downstream projects using the internals all the time.

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jan 30, 2024
nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jan 30, 2024
@cjw296
Copy link

cjw296 commented Jan 31, 2024

I completely understand the feeling, and it is frustrating to us as well... believe us, we don't make light to introduce breaking changes, even for internal APIs -- @bluetech goes to great pains grepping a large number of plugins to check if internal changes are breaking them, for example.

Perhaps @bluetech could add Sybil to the plugins checked? I've lost count of the number of times I've had new release of pytest break Sybil because it has to use "non-public" APS...

@nicoddemus
Copy link
Member

@cjw296 that sounds like a great idea! Feel free to open a PR in that regard to get the ball rolling if you want.

@cjw296
Copy link

cjw296 commented Jan 31, 2024

@nicoddemus - I have no idea what you're repo you're looking for me to open a PR against or what you're expecting it to contain...

elliotgunton added a commit to argoproj-labs/hera that referenced this issue Feb 7, 2024
* Pytest 8 breaks when using the asyncio library, see issue
pytest-dev/pytest#11868
* We should use pytest-asyncio instead of the asyncio library directly
once pytest-asyncio supports pytest >=8

Signed-off-by: Elliot Gunton <egunton@bloomberg.net>
elliotgunton added a commit to argoproj-labs/hera that referenced this issue Feb 7, 2024
Limits [pytest](https://github.com/pytest-dev/pytest) to <8.0.0.
* Pytest 8 breaks when using the asyncio library, see issue
pytest-dev/pytest#11868
* We should use pytest-asyncio instead of the asyncio library directly
once pytest-asyncio supports pytest >=8

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Elliot Gunton <egunton@bloomberg.net>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Elliot Gunton <egunton@bloomberg.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet