Implement PEP 649 and PEP 749

[JelleZijlstra]

PEP-649 has been accepted and should be implemented in Python 3.14. Let's use this issue to track the implementation:

• Decide on or clarify the adoption strategy (https://discuss.python.org/t/pep-649-deferred-evaluation-of-annotations-tentatively-accepted/21331/44)
• Add new __annotate__ attributes gh-119180: PEP 649: Add __annotate__ attributes #119209
• Implement core interpreter changes (e.g., new symbol table functionality) gh-119180: PEP 649 compiler changes #119361
• Implement new Python-level APIs, like the new format argument to inspect.get_annotations gh-119180: Add annotationlib module to support PEP 649 #119891
  • (PEP-649 specifies that inspect.AnnotationsFormat should be a "global enum". Is that desirable? TBD. https://github.com/python/cpython/pull/119361/files#r1614753031)
• Document the new semantics gh-119180: Documentation for PEP 649 and 749 #122235
  • Add higher-level documentation, e.g. a migration guide away from from __future__ import annotations; an introduction to annotationlib; an update to Larry's annotations HOWTO
  • The SC would like https://peps.python.org/pep-0749/#appendix incorporated into the docs
• Make staticmethod and classmethod not force evaluation of annotations gh-119180: Lazily wrap annotations on classmethod and staticmethod #119864
• Adapt standard library code that uses annotations to work excellently with PEP 649
  • dataclasses gh-119180: Add annotationlib module to support PEP 649 #119891
  • typing.py: TypedDict, NamedTuple, try to remove dependency on inspect. (If it can't be removed, get rid of the awkward dance we do for typing.Protocol to import inspect.getattr_static lazily.)
  • singledispatch gh-119180: Add annotationlib module to support PEP 649 #119891
    • Also make it only evaluate the relevant parameter, gh-119180: Add annotationlib module to support PEP 649 #119891 (comment)
  • functools.update_wrapper gh-119180: Add annotationlib module to support PEP 649 #119891, PEP 649: functools.update_wrapper implementation #124342
• Add PEP 649-like functionality to TypeAliasType, TypeVar, etc. (pep649-typevar branch in my fork)
• Make sure the ecosystem is ready by testing third-party libraries like Pydantic
  • typing_extensions: needs changes similar to those in typing.py (Support my PEP 649 branch typing_extensions#412)
  • beartype: had various unrelated problems on 3.13 ([Future] Python 3.13 conformance beartype/beartype#387); a few additional tests fail on the PR branch due to direct manipulation of __dict__
  • typeguard: some unrelated 3.13 problems (Support Python 3.13 agronholm/typeguard#460); no test failures related to PEP 649
  • pyanalyze: some unrelated 3.13 issues and a few changes necessary due to direct __dict__ access (Support Python 3.13 quora/pyanalyze#773)
  • pydantic: failed to build for 3.14 so far
• Optimize the implementation. Ideas:
  • Avoid creation of function objects and store only code objects instead PEP 649: Avoid creation of function objects for __annotate__ #124157
  • Avoid creation of AST nodes and use of eval() for ForwardRefs that are just names gh-119180: Avoid going through AST and eval() when possible in annotationlib #124337

Things to revisit:

• Should FORWARDREF be the default? Some improvements for Python 3.14 / PEP 649 support beartype/beartype#440 (comment)
• Add VALUE_WITH_FAKE_GLOBALS format
• Name of __annotate__ parameter
• Should setting __annotations__ invalidate __annotate__?
• Rename SOURCE to STRING?

I am planning to work on the interpreter core first.

cc @larryhastings @carljm @samuelcolvin

Linked PRs

• gh-119180: PEP 649: Add __annotate__ attributes #119209
• gh-119180: Add LOAD_COMMON_CONSTANT opcode #119321
• gh-119180: PEP 649 compiler changes #119361
• gh-119180: Update the magic number #119397
• gh-119180: Lazily wrap annotations on classmethod and staticmethod #119864
• gh-119180: Add annotationlib module to support PEP 649 #119891
• gh-119180: Fix annotations lookup on classes with custom metaclasses #120719
• gh-119180: Alternative approach to metaclass annotations: never use the dict #120816
• gh-119180: Use type descriptors to access annotations (PEP 749) #122074
• gh-119180: Improvements to ForwardRef.evaluate #122210
• gh-119180: Add evaluate functions for type params and type aliases #122212
• gh-119180: Documentation for PEP 649 and 749 #122235
• gh-119180: annotationlib: Fix __all__, other minor fixes #122365
• gh-119180: Add myself as CODEOWNER for annotationlib #122366
• gh-119180: Fix annotationlib.ForwardRef.evaluate with no globals #124326
• gh-119180: Avoid going through AST and eval() when possible in annotationlib #124337
• gh-119180: Add discussion of annotations to the 3.14 What's New #124393
• gh-119180: Add VALUE_WITH_FAKE_GLOBALS format to annotationlib #124415
• gh-119180: Rename parameter to __annotate__ functions #124461
• gh-119180: Make FORWARDREF format look at __annotations__ first #124479
• gh-119180: Disallow instantiation of ConstEvaluator objects #124561
• gh-119180: Rename SOURCE format to STRING #124620
• gh-119180: No longer set __annotations__ in __main__ #124634
• gh-119180: Set the name of the param to __annotate__ to "format" #124730
• gh-119180: Use equality when comparing against annotationlib.Format #131755
• gh-119180: Make the FORWARDREF format work with more kinds of runtime errors #133407
• gh-119180: More documentation for PEP 649/749 #133552
• gh-119180: annotationlib: Fix values of Format members in docs #133841
• [3.14] gh-119180: More documentation for PEP 649/749 (GH-133552) #133902
• [3.14] gh-119180: annotationlib: Fix values of Format members in docs (GH-133841) #133903

[AlexWaygood]

Just a cosmetic inconsistency I noticed: comparisons with annotationslib.Format are done using both equality and identity:

The enum docs recommend that comparisons like this should be done using identity since each enum member is a singleton, but I don't think it's a huge issue

[Viicos]

Yes iirc only an issue when you reload modules or something like this. This can be ignored, just wanted to notify it in case we want proper consistency here :)

[JelleZijlstra]

We should use == since the integer values may be used interchangeably (they're what gets used by some low-level code). I'd accept a PR to change to ==.

[JelleZijlstra]

PEP 749 has been accepted and the implementation on main is done as far as I'm concerned, though I'm sure we'll discover more things to tweak.

We need to add more docs; I'll hopefully work on that soon.