Failures with Python 3.13: TypeError: argument list must be a tuple

[hroncok]

Python Version

3.13.0a2

pytest Version

7.4.3

Package Version

2.13.0 (main)

Description

Hello. In Fedora, we are trying to build everything with pre-releases of Python 3.13. Currently, we have 3.13.0a2.

Many projects have moved from freezegun to time-machine and we cannot test them unless we have a working time-machine first.

To reproduce the test failures we see in Fedora, I cloned this repository and did:

$ cp requirements/py31{2,3}.txt
$ tox -e py313
.pkg-cpython313: _optional_hooks> python .../time-machine/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython313: get_requires_for_build_wheel> python .../time-machine/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython313: build_wheel> python .../time-machine/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py313: install_package> python -I -m pip install --force-reinstall --no-deps .../time-machine/.tox/.tmp/package/3/time_machine-2.13.0-cp313-cp313-linux_x86_64.whl
py313: commands[0]> python -W error::ResourceWarning -W error::DeprecationWarning -W error::PendingDeprecationWarning -W ignore:datetime.datetime.utcfromtimestamp:DeprecationWarning -W ignore:datetime.datetime.utcnow:DeprecationWarning -m coverage run -m pytest tests
============================= test session starts ==============================
platform linux -- Python 3.13.0a2, pytest-7.4.3, pluggy-1.3.0
cachedir: .tox/py313/.pytest_cache
Using --randomly-seed=2081647715
rootdir: .../time-machine
configfile: pyproject.toml
plugins: time-machine-2.13.0, randomly-3.15.0
collected 96 items

tests/test_time_machine.py ....................................FF......F [ 46%]
.....F.............................sF.........F....                      [100%]

=================================== FAILURES ===================================
_______________________ test_time_clock_gettime_realtime _______________________

    @py_have_clock_gettime
    def test_time_clock_gettime_realtime():
        with time_machine.travel(EPOCH + 180.0):
>           now = time.clock_gettime(time.CLOCK_REALTIME)
E           TypeError: argument list must be a tuple

tests/test_time_machine.py:150: TypeError
_____________________ test_time_clock_gettime_ns_realtime ______________________

    @py_have_clock_gettime
    def test_time_clock_gettime_ns_realtime():
        with time_machine.travel(EPOCH + 190.0):
>           first = time.clock_gettime_ns(time.CLOCK_REALTIME)
E           TypeError: argument list must be a tuple

tests/test_time_machine.py:177: TypeError
_________________ test_time_clock_gettime_monotonic_unaffected _________________

    @py_have_clock_gettime
    def test_time_clock_gettime_monotonic_unaffected():
>       start = time.clock_gettime(time.CLOCK_MONOTONIC)
E       TypeError: argument list must be a tuple

tests/test_time_machine.py:161: TypeError
_______________ test_time_clock_gettime_ns_monotonic_unaffected ________________

    @py_have_clock_gettime
    def test_time_clock_gettime_ns_monotonic_unaffected():
>       start = time.clock_gettime_ns(time.CLOCK_MONOTONIC)
E       TypeError: argument list must be a tuple

tests/test_time_machine.py:191: TypeError
___________________ TestEscapeHatch.test_time_clock_gettime ____________________

self = <tests.test_time_machine.TestEscapeHatch object at 0x7f5d5207e310>

    @py_have_clock_gettime
    def test_time_clock_gettime(self):
>       now = time.clock_gettime(time.CLOCK_REALTIME)
E       TypeError: argument list must be a tuple

tests/test_time_machine.py:838: TypeError
__________________ TestEscapeHatch.test_time_clock_gettime_ns __________________

self = <tests.test_time_machine.TestEscapeHatch object at 0x7f5d5207e630>

    @py_have_clock_gettime
    def test_time_clock_gettime_ns(self):
>       now = time.clock_gettime_ns(time.CLOCK_REALTIME)
E       TypeError: argument list must be a tuple

tests/test_time_machine.py:846: TypeError
=========================== short test summary info ============================
FAILED tests/test_time_machine.py::test_time_clock_gettime_realtime - TypeErr...
FAILED tests/test_time_machine.py::test_time_clock_gettime_ns_realtime - Type...
FAILED tests/test_time_machine.py::test_time_clock_gettime_monotonic_unaffected
FAILED tests/test_time_machine.py::test_time_clock_gettime_ns_monotonic_unaffected
FAILED tests/test_time_machine.py::TestEscapeHatch::test_time_clock_gettime
FAILED tests/test_time_machine.py::TestEscapeHatch::test_time_clock_gettime_ns
=================== 6 failed, 89 passed, 1 skipped in 0.76s ====================
.../time-machine/.tox/py313/lib64/python3.13/site-packages/coverage/inorout.py:507: CoverageWarning: Module src/_time_machine.c was never imported. (module-not-imported)
  self.warn(f"Module {pkg} was never imported.", slug="module-not-imported")
py313: exit 1 (1.47 seconds) .../time-machine> python -W error::ResourceWarning -W error::DeprecationWarning -W error::PendingDeprecationWarning -W ignore:datetime.datetime.utcfromtimestamp:DeprecationWarning -W ignore:datetime.datetime.utcnow:DeprecationWarning -m coverage run -m pytest tests pid=3348698
.pkg-cpython313: _exit> python .../time-machine/.tox/.tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  py313: FAIL code 1 (2.46=setup[0.99]+cmd[1.47] seconds)
  evaluation failed :( (2.56 seconds)

[adamchainz]

I haven't done any work on 3.13 yet. Normally I wait until the beta phase to test, since there’s otherwise a lot of churn (as you already noted with this vectorcall thing).

If Fedora needs this sooner, please open a PR to test on 3.13 and try to fix the issues. (Also, consider sponsoring... 😉).

[hroncok]

I edited my report, reinstalling Python 3.13 solved the _PyObject_Vectorcall problem, I must have some weird combo of a1 and a2 installed.

[hroncok]

I will try fixing the failures, but I open this issue anyway in case you have an idea right now what might be the reason. I don't have the authority to sponsor on behalf of Fedora, sorry.

[adamchainz]

Maybe you can buy my books with your learning budget!

[hroncok]

I suspect the failure is caused by python/cpython@4fe22c73770

The time-machine module replaces functions from the CPython time module. The replaced CPython functions might no longer take arg tuples.

The are now METH_O

[hroncok]

I have code that works with 3.13. Will make it backward compatible now.

[vstinner]

This code to override time attributes looks fragile:

    PyCFunctionObject *time_clock_gettime_ns = (PyCFunctionObject *) PyObject_GetAttrString(time_module, "clock_gettime_ns");
    if (time_clock_gettime_ns != NULL) {
        state->original_clock_gettime_ns = time_clock_gettime_ns->m_ml->ml_meth;
        time_clock_gettime_ns->m_ml->ml_meth = _time_machine_clock_gettime_ns;
        Py_DECREF(time_clock_gettime_ns);
    }

Why not just calling PyObject_SetAttrString(time_module, "clock_gettime_ns", new_function)?

[adamchainz]

The pointer replacement is the whole point of time-machine. Replacing the module level function doesn’t affect imports like from time import time.

[vstinner]

The pointer replacement is the whole point of time-machine. Replacing the module level function doesn’t affect imports like from time import time.

If you monkey-patch the module, you can do it early. Or you can override the libc function with LD_PRELOAD.

If you prefer to modify time_clock_gettime_ns->m_ml->ml_meth, you should just use the same calling convention than the Python 3.13 calling convention of time.clock_gettime_ns(). And maybe also add a check at runtime to avoid such bad surprise.

[adamchainz]

If you monkey-patch the module, you can do it early. Or you can override the libc function with LD_PRELOAD.

The whole inspiration of time-machine is that those mechanisms are insufficient. It’s really hard or impossible to monkey-patch early enough and LD_PRELOAD is not cross-platform. See the introductory post.

I’d be interested to know about other possible techniques if you have any!

If you prefer to modify time_clock_gettime_ns->m_ml->ml_meth, you should just use the same calling convention than the Python 3.13 calling convention of time.clock_gettime_ns(). And maybe also add a check at runtime to avoid such bad surprise.

Indeed, this is what all the other methods do. There were previously forks for calling convention changes until Python 3.7 support was dropped, see 6ca4673, ab7a388.