Fixes for Python 3.14 and PEP 649

[JelleZijlstra]

I ran the typeguard test suite on the current CPython main branch to
look for possible breakage from PEP 649 and 749, which makes large
changes to how annotations work.

I found three problems:

• DeprecationWarnings from uses of ForwardRef._evaluate, which we're
  deprecating. Made a change to use the new public API for ForwardRef.evaluate
  instead.
• A test failure that showed "annotationlib" as the module name for a forward
  ref. I made it so that it doesn't use module for ForwardRef objects.
• One remaining test failure that I haven't tracked down:
  tests/test_instrumentation.py::test_typevar_forwardref[importhook]
  It's apparently due to caching (it doesn't fail if I run only that one test),
  but I haven't tracked down the exact cause.

I also added some new tests relying on 3.14-only behavior (unquoted annotations containing forward references). Those mostly worked fine, but I had to make another tweak to how functions are wrapped.

Since the APIs in Python 3.14 may still change, I'll make this a draft
PR for now.

Checklist

If this is a user-facing code change, like a bugfix or a new feature, please ensure that
you've fulfilled the following conditions (where applicable):

• You've added tests (in tests/) added which would fail without your patch
• You've updated the documentation (in docs/, in case of behavior changes or new
  features)
• You've added a new changelog entry (in docs/versionhistory.rst).

[coveralls]

coverage: 94.593% (+0.09%) from 94.507%
when pulling 739ec42 on JelleZijlstra:fix314
into f282802 on agronholm:master.

[musicinmybrain]

Thanks for working on this! I just tested this branch with Python 3.14.0a1, and it looks like some further adjustments will be required:

========================================================================================== short test summary info ==========================================================================================
FAILED tests/test_instrumentation.py::test_typevar_forwardref[importhook] - typeguard.TypeCheckError: argument "x" (class dummymodule.DummyClass) is not a subclass of dummymodule.DummyClass
ERROR tests/test_instrumentation.py::TestUsesForwardRef::test_success[typechecked] - ModuleNotFoundError: No module named 'deferredannos'
ERROR tests/test_instrumentation.py::TestUsesForwardRef::test_failure[typechecked] - ModuleNotFoundError: No module named 'deferredannos'
ERROR tests/test_instrumentation.py::TestUsesForwardRef::test_success[importhook] - ModuleNotFoundError: No module named 'deferredannos'
ERROR tests/test_instrumentation.py::TestUsesForwardRef::test_failure[importhook] - ModuleNotFoundError: No module named 'deferredannos'
======================================================================= 1 failed, 474 passed, 4 skipped, 9 xfailed, 4 errors in 2.13s =======================================================================

[agronholm]

@JelleZijlstra it looks like the tests are passing, so is there any reason this shouldn't be merged?
Dang, I forgot we're not testing against 3.14 in CI.

[agronholm]

@JelleZijlstra did you forget to commit the deferredannos module in the test suite?

[agronholm]

I looked into the one remaining actual failure. The subclass check fails because the value and the class, albeit both classes from the same supposed origin, have different identities. I'm not sure yet why this happens only on 3.14.

[agronholm]

Interesting – on 3.13, the subclass check isn't happening at all. I'm continuing to investigate.

[agronholm]

As this clearly has something to do with the forwardref changes in 3.14, I decided to add some debug output which showed a key difference between 3.13 and 3.14.

Here's a test run of test_typevar_forwardref on 3.13:

checking value=<class 'dummymodule.DummyClass'> (id=5592c6224090)
evaluated ForwardRef('DummyClass') to <class 'dummymodule.DummyClass'> (id=5592c6224090)
evaluated ForwardRef('DummyClass') to <class 'dummymodule.DummyClass'> (id=5592c6224090)
checking value=<class 'dummymodule.DummyClass'> (id=5592c6232e20)
evaluated ForwardRef('DummyClass') to <class 'dummymodule.DummyClass'> (id=5592c6232e20)
evaluated ForwardRef('DummyClass') to <class 'dummymodule.DummyClass'> (id=5592c6232e20)

And on 3.14:

checking value=<class 'dummymodule.DummyClass'> (id=55d1660a5030)
evaluated ForwardRef('DummyClass') to <class 'dummymodule.DummyClass'> (id=55d1660a5030)
evaluated ForwardRef('DummyClass') to <class 'dummymodule.DummyClass'> (id=55d1660a5030)
checking value=<class 'dummymodule.DummyClass'> (id=55d1660a96d0)
evaluated ForwardRef('DummyClass') to <class 'dummymodule.DummyClass'> (id=55d1660a5030)

As you can see, the forwardref is evaluated to the same object at all times on v3.14, while on v3.13 it evaluates to a different object, as expected since dummymodule is reloaded.

[JelleZijlstra]

I see, this is probably because I made ForwardRef cache its value more aggressively. On 3.13, it caches __forward_value__, but ignores the cache if localns is not globalns for some reason: https://github.com/python/cpython/blob/21200895478b8076fc0bda6ee2c26f0d87c9404b/Lib/typing.py#L1053

On 3.14 I instead made it always use the cached value: https://github.com/python/cpython/blob/bbeb219354764aef85e660be6570f0f329e7227e/Lib/annotationlib.py#L100

The old behavior wasn't documented and it feels inconsistent, but perhaps the new caching behavior is too aggressive. I wonder if it would make sense to not cache the values at all?

(Also, sorry for the missing file. I think I lost the local branch for this PR, I'll have to take a look and try to recreate it.)

[agronholm]

Something else bugs me: how can the ForwardRef object be the same object on py3.14 even after I've reloaded the module? I've confirmed that the ID of the function I'm calling is different, as is dummymodule.DummyClass.

id of imported module: 7f0620136f70
id of typevar_forwardref: 7f0620142770
checking value=<class 'dummymodule.DummyClass'> (id=564aadc496d0)
evaluated ForwardRef('DummyClass') (id=7f062004bda0) to <class 'dummymodule.DummyClass'> (id=564aadc496d0) with memo 7f061ffb73c0
evaluated ForwardRef('DummyClass') (id=7f062004bda0) to <class 'dummymodule.DummyClass'> (id=564aadc496d0) with memo 7f061ffb73c0
id of imported module: 7f0620136f70
id of typevar_forwardref: 7f0620142560
checking value=<class 'dummymodule.DummyClass'> (id=564aadc4f2f0)
evaluated ForwardRef('DummyClass') (id=7f062004bda0) to <class 'dummymodule.DummyClass'> (id=564aadc496d0) with memo 7f06202aec80

[JelleZijlstra]

Where do you get the ForwardRef object from? There are various kinds of caching in typing.py. Maybe this is python/cpython#128593.

[agronholm]

I'll debug this further to answer that question.

[agronholm]

This took me way too long to figure out, but here's where it goes wrong:

        annotation = (
            Type[origin_type.__bound__] if subclass_check else origin_type.__bound__
        )

For some bizarre reason, the type parameter for the resulting Type[] is NOT always the same ForwardRef object as origin_type.__bound__! But if I use the built-in type instead of typing.Type, then it behaves as expected. This also fixes the test on Python 3.14.

Now I'll just need the deferredannos module to get the rest of the tests passing. @JelleZijlstra any progress on that?

EDIT: I forgot to mention that Type[...] seemed to dig up the old ForwardRef object from before the module was reloaded, and I have no clue where from, or why. I could not step into it with the debugger either as this was apparently C code.

[JelleZijlstra]

For some bizarre reason, the type parameter for the resulting Type[] is NOT always the same ForwardRef object as origin_type.bound! But if I use the built-in type instead of typing.Type, then it behaves as expected. This also fixes the test on Python 3.14.

I think this is because there's a typing._tp_cache on Type.__getitem__, and ForwardRef has __hash__ and __eq__ that look at only some attributes (already on 3.13: https://github.com/python/cpython/blob/8b4a0d641cde667f94ce49f5e64da6bd9d6fbd9c/Lib/typing.py#L1096). I am going to open an issue on CPython to remove that behavior; I think it makes more sense to compare ForwardRefs by identity.

Now I'll just need the deferredannos module

Just pushed it, sorry for that!

[agronholm]

I've added 3.14 to the CI, and the tests seem to pass. Perhaps it's time to remove the draft status, unless you have something more to add?

[musicinmybrain]

I tried this as a patch for the 4.4.1 release in Fedora’s python-typeguard package – rebased on 4.4.1, and with documentation changes omitted to keep the patch clean – and it built with tests passing for Python 3.14.0a4. So you can consider that a vote of confidence.

[JelleZijlstra]

Yes, taken out of draft!

One caveat: Python 3.14 is still in alpha, and PEP 749 has not yet been accepted, so some of the aspects of 3.14 might still change before the final release. Hopefully not too much!

[agronholm]

Thanks!