Test failure with Python 3.13: TestUpdateAbstractMethods.test_remain_abstract[False] does not raise the expected TypeError

[befeleme]

attrs: 23.2.0 built in Fedora Linux 40 with Python 3.13.0a3 cause this test failure:

____________ TestUpdateAbstractMethods.test_remain_abstract[False] _____________

self = <tests.test_abc.TestUpdateAbstractMethods object at 0x7f8f99690860>
slots = False

    def test_remain_abstract(self, slots):
        """
        If an attrs class inherits from an abstract class but doesn't implement
        abstract methods, it remains abstract.
        """
    
        class A(abc.ABC):
            @abc.abstractmethod
            def foo(self):
                pass
    
        @attrs.define(slots=slots)
        class StillAbstract(A):
            pass
    
        assert inspect.isabstract(StillAbstract)
        expected_exception_message = (
            "^Can't instantiate abstract class StillAbstract without an "
            "implementation for abstract method 'foo'$"
            if PY_3_12_PLUS
            else "class StillAbstract with abstract method foo"
        )
>       with pytest.raises(TypeError, match=expected_exception_message):
E       Failed: DID NOT RAISE <class 'TypeError'>

tests/test_abc.py:59: Failed

[webknjaz]

@befeleme What does it raise instead? Do you have a more complete excerpt? What happens if you drop with pytest.raises() and just run what's inside?

[befeleme]

I'm running python -m pytest -k "test_remain_abstract" -vvv in a venv with Python 3.13.0a3 and the project configured according to the contributors guide.

It gets weirder, now slots = False passes, but slots = True fails:

________________________________ TestUpdateAbstractMethods.test_remain_abstract[True] _________________________________

self = <tests.test_abc.TestUpdateAbstractMethods object at 0x7f32a2d892b0>, slots = True

    def test_remain_abstract(self, slots):
        """
        If an attrs class inherits from an abstract class but doesn't implement
        abstract methods, it remains abstract.
        """
    
        class A(abc.ABC):
            @abc.abstractmethod
            def foo(self):
                pass
    
        @attrs.define(slots=slots)
        class StillAbstract(A):
            pass
    
        assert inspect.isabstract(StillAbstract)
        expected_exception_message = (
            "^Can't instantiate abstract class StillAbstract without an "
            "implementation for abstract method 'foo'$"
            if PY_3_12_PLUS
            else "class StillAbstract with abstract method foo"
        )
        # with pytest.raises(TypeError, match=expected_exception_message):
>       StillAbstract()
E       TypeError: Can't instantiate abstract class StillAbstract without an implementation for abstract method 'foo'

tests/test_abc.py:60: TypeError

[befeleme]

In an interactive Python 3.13.0a3 console in both cases TypeError is raised. I'm really confused.

[webknjaz]

This is reproducible with 3.13.0a3 and is not Fedora-specific (I'm using pyenv under Gentoo Linux).
Here's another observation: running each of the parameterized tests separately succeeds and running both in the same test session makes one fail. There must be a shared state or caching somewhere.

$ tox -qq -e py-tests -- tests/test_abc.py::TestUpdateAbstractMethods::test_remain_abstract[False]
===================================================================== test session starts =====================================================================
platform linux -- Python 3.13.0a3, pytest-8.0.0, pluggy-1.4.0
cachedir: .tox/py-tests/.pytest_cache
rootdir: ~/src/github/python-attrs/attrs
configfile: pyproject.toml
plugins: hypothesis-6.97.2, xdist-3.5.0, mypy-plugins-3.0.0
collected 1 item

tests/test_abc.py .                                                                                                                                     [100%]

====================================================================== 1 passed in 0.02s ======================================================================

$ tox -qq -e py-tests -- tests/test_abc.py::TestUpdateAbstractMethods::test_remain_abstract[True]
===================================================================== test session starts =====================================================================
platform linux -- Python 3.13.0a3, pytest-8.0.0, pluggy-1.4.0
cachedir: .tox/py-tests/.pytest_cache
rootdir: ~/src/github/python-attrs/attrs
configfile: pyproject.toml
plugins: hypothesis-6.97.2, xdist-3.5.0, mypy-plugins-3.0.0
collected 1 item

tests/test_abc.py .                                                                                                                                     [100%]

====================================================================== 1 passed in 0.02s ======================================================================

$ tox -qq -e py-tests -- tests/test_abc.py::TestUpdateAbstractMethods::test_remain_abstract
===================================================================== test session starts =====================================================================
platform linux -- Python 3.13.0a3, pytest-8.0.0, pluggy-1.4.0
cachedir: .tox/py-tests/.pytest_cache
rootdir: ~/src/github/python-attrs/attrs
configfile: pyproject.toml
plugins: hypothesis-6.97.2, xdist-3.5.0, mypy-plugins-3.0.0
collected 2 items

tests/test_abc.py .F                                                                                                                                    [100%]

========================================================================== FAILURES ===========================================================================
____________________________________________________ TestUpdateAbstractMethods.test_remain_abstract[True] _____________________________________________________

self = <tests.test_abc.TestUpdateAbstractMethods object at 0x7f1901a874d0>, slots = True

    def test_remain_abstract(self, slots):
        """
        If an attrs class inherits from an abstract class but doesn't implement
        abstract methods, it remains abstract.
        """
    
        class A(abc.ABC):
            @abc.abstractmethod
            def foo(self):
                pass
    
        @attrs.define(slots=slots)
        class StillAbstract(A):
            pass
    
        assert inspect.isabstract(StillAbstract)
        expected_exception_message = (
            "^Can't instantiate abstract class StillAbstract without an "
            "implementation for abstract method 'foo'$"
            if PY_3_12_PLUS
            else "class StillAbstract with abstract method foo"
        )
>       with pytest.raises(TypeError, match=expected_exception_message):
E       Failed: DID NOT RAISE <class 'TypeError'>

tests/test_abc.py:59: Failed
=================================================================== short test summary info ===================================================================
FAILED tests/test_abc.py::TestUpdateAbstractMethods::test_remain_abstract[True] - Failed: DID NOT RAISE <class 'TypeError'>
================================================================= 1 failed, 1 passed in 0.04s =================================================================

Action items: bisect when this started happening. Is 3.13.0a2 working? What's the last release where testing succeeds? Is it CPython that needs fixing or pytest, maybe? I'd look into the slots changes in CPython, I suppose.

[hynek]

I see you run pytest 8 – could this be actually related to #1233? Would you mind re-testing with the pin I've added the other day?

[webknjaz]

@hynek I tested with older pytest versions too, just forgot to mention those. Perhaps it's time to plug py313 into the CI as experimental.

[webknjaz]

Also, in #1233, the log linked has tests/test_abc.py::TestUpdateAbstractMethods::test_remain_abstract succeeding, so I'm pretty sure this one is a separate problem.

[frenzymadness]

I'm not sure what happened and where but during my work on a different issue with Python 3.13 I've realized that I'm no longer able to reproduce this one. Using Python 3.13.0a4 and pytest 8.0.2.

Could please anybody verify it?

[hynek]

I can confirm that all of test_abc is passing now! 🎉

I'm opening #1256 to see if we can add it to CI.