diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index cad8dd7db7228ad..75a784f620f5fde 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -59,36 +59,106 @@ Summary -- Release highlights .. This section singles out the most important changes in Python 3.12. Brevity is key. +Python 3.12 is the latest stable release of the Python programming language, +with a mix of changes to the language and the standard library. +The library changes focus on cleaning up deprecated APIs, usability, and correctness. +Of note, the :mod:`!distutils` package has been removed from the standard library. +Filesystem support in :mod:`os` and :mod:`pathlib` has seen a number of improvements, +and several modules have better performance. + +The language changes focus on usability, +as :term:`f-strings ` have had many limitations removed +and 'Did you mean ...' suggestions continue to improve. +The new :ref:`type parameter syntax ` +and :keyword:`type` statement improve ergonomics for using :term:`generic types +` and :term:`type aliases ` with static type checkers. + +This article doesn't attempt to provide a complete specification of all new features, +but instead gives a convenient overview. +For full details, you should refer to the documentation, +such as the :ref:`Library Reference ` +and :ref:`Language Reference `. +If you want to understand the complete implementation and design rationale for a change, +refer to the PEP for a particular new feature; +but note that PEPs usually are not kept up-to-date +once a feature has been fully implemented. + +-------------- .. PEP-sized items next. +New syntax features: + +* :ref:`PEP 695 `, type parameter syntax and the :keyword:`type` statement + New grammar features: -* :ref:`whatsnew312-pep701` +* :ref:`PEP 701 `, :term:`f-strings ` in the grammar Interpreter improvements: -* :ref:`whatsnew312-pep684` +* :ref:`PEP 684 `, a unique per-interpreter :term:`GIL + ` +* :ref:`PEP 669 `, low impact monitoring +* `Improved 'Did you mean ...' suggestions `_ + for :exc:`NameError`, :exc:`ImportError`, and :exc:`SyntaxError` exceptions -* :ref:`whatsnew312-pep669` +Python data model improvements: -New typing features: +* :ref:`PEP 688 `, using the :ref:`buffer protocol + ` from Python + +Significant improvements in the standard library: + +* The :class:`pathlib.Path` class now supports subclassing +* The :mod:`os` module received several improvements for Windows support +* A :ref:`command-line interface ` has been added to the + :mod:`sqlite3` module +* :func:`isinstance` checks against :func:`runtime-checkable protocols + ` enjoy a speed up of between two and 20 times +* The :mod:`asyncio` package has had a number of performance improvements, + with some benchmarks showing a 75% speed up. +* A :ref:`command-line interface ` has been added to the + :mod:`uuid` module +* Due to the changes in :ref:`PEP 701 `, + producing tokens via the :mod:`tokenize` module is up to up to 64% faster. + +Security improvements: -* :ref:`whatsnew312-pep688` +* Replace the builtin :mod:`hashlib` implementations of + SHA1, SHA3, SHA2-384, SHA2-512, and MD5 with formally verified code from the + `HACL* `__ project. + These builtin implementations remain as fallbacks that are only used when + OpenSSL does not provide them. -* :ref:`whatsnew312-pep692` +C API improvements: -* :ref:`whatsnew312-pep695` +* :ref:`PEP 697 `, unstable C API tier +* :ref:`PEP 683 `, immortal objects -* :ref:`whatsnew312-pep698` +CPython implementation improvements: + +* :ref:`PEP 709 `, comprehension inlining +* :ref:`CPython support ` for the Linux ``perf`` profiler +* Implement stack overflow protection on supported platforms + +New typing features: + +* :ref:`PEP 692 `, using :class:`~typing.TypedDict` to + annotate :term:`**kwargs ` +* :ref:`PEP 698 `, :func:`typing.override` decorator Important deprecations, removals or restrictions: -* :pep:`623`: Remove wstr from Unicode +* :pep:`623`: Remove ``wstr`` from Unicode objects in Python's C API, + reducing the size of every :class:`str` object by at least 8 bytes. -* :pep:`632`: Remove the ``distutils`` package. See - `the migration guide `_ - for advice on its replacement. +* :pep:`632`: Remove the :mod:`!distutils` package. + See `the migration guide `_ + for advice replacing the APIs it provided. + The third-party `Setuptools `__ + package continues to provide :mod:`!distutils`, + if you still require it in Python 3.12 and beyond. * :gh:`95299`: Do not pre-install ``setuptools`` in virtual environments created with :mod:`venv`. @@ -97,60 +167,77 @@ Important deprecations, removals or restrictions: run ``pip install setuptools`` in the :ref:`activated ` virtual environment. -Improved Error Messages -======================= +* The :mod:`!asynchat`, :mod:`!asyncore`, and :mod:`!imp` modules have been + removed, along with several :class:`unittest.TestCase` + `method aliases `_. -* Modules from the standard library are now potentially suggested as part of - the error messages displayed by the interpreter when a :exc:`NameError` is - raised to the top level. (Contributed by Pablo Galindo in :gh:`98254`.) - >>> sys.version_info - Traceback (most recent call last): - File "", line 1, in - NameError: name 'sys' is not defined. Did you forget to import 'sys'? +New Features +============ -* Improve the error suggestion for :exc:`NameError` exceptions for instances. - Now if a :exc:`NameError` is raised in a method and the instance has an - attribute that's exactly equal to the name in the exception, the suggestion - will include ``self.`` instead of the closest match in the method - scope. (Contributed by Pablo Galindo in :gh:`99139`.) +.. _whatsnew312-pep695: - >>> class A: - ... def __init__(self): - ... self.blech = 1 - ... - ... def foo(self): - ... somethin = blech - ... - >>> A().foo() - Traceback (most recent call last): - File "", line 1 - somethin = blech - ^^^^^ - NameError: name 'blech' is not defined. Did you mean: 'self.blech'? +PEP 695: Type Parameter Syntax +------------------------------ -* Improve the :exc:`SyntaxError` error message when the user types ``import x - from y`` instead of ``from y import x``. (Contributed by Pablo Galindo in :gh:`98931`.) +Generic classes and functions under :pep:`484` were declared using a verbose syntax +that left the scope of type parameters unclear and required explicit declarations of +variance. - >>> import a.y.z from b.y.z - Traceback (most recent call last): - File "", line 1 - import a.y.z from b.y.z - ^^^^^^^^^^^^^^^^^^^^^^^ - SyntaxError: Did you mean to use 'from ... import ...' instead? +:pep:`695` introduces a new, more compact and explicit way to create +:ref:`generic classes ` and :ref:`functions `:: -* :exc:`ImportError` exceptions raised from failed ``from import - `` statements now include suggestions for the value of ```` based on the - available names in ````. (Contributed by Pablo Galindo in :gh:`91058`.) + def max[T](args: Iterable[T]) -> T: + ... - >>> from collections import chainmap - Traceback (most recent call last): - File "", line 1, in - ImportError: cannot import name 'chainmap' from 'collections'. Did you mean: 'ChainMap'? + class list[T]: + def __getitem__(self, index: int, /) -> T: + ... + def append(self, element: T) -> None: + ... -New Features -============ +In addition, the PEP introduces a new way to declare :ref:`type aliases ` +using the :keyword:`type` statement, which creates an instance of +:class:`~typing.TypeAliasType`:: + + type Point = tuple[float, float] + +Type aliases can also be :ref:`generic `:: + + type Point[T] = tuple[T, T] + +The new syntax allows declaring :class:`~typing.TypeVarTuple` +and :class:`~typing.ParamSpec` parameters, as well as :class:`~typing.TypeVar` +parameters with bounds or constraints:: + + type IntFunc[**P] = Callable[P, int] # ParamSpec + type LabeledTuple[*Ts] = tuple[str, *Ts] # TypeVarTuple + type HashableSequence[T: Hashable] = Sequence[T] # TypeVar with bound + type IntOrStrSequence[T: (int, str)] = Sequence[T] # TypeVar with constraints + +The value of type aliases and the bound and constraints of type variables +created through this syntax are evaluated only on demand (see +:ref:`lazy evaluation `). This means type aliases are able to +refer to other types defined later in the file. + +Type parameters declared through a type parameter list are visible within the +scope of the declaration and any nested scopes, but not in the outer scope. For +example, they can be used in the type annotations for the methods of a generic +class or in the class body. However, they cannot be used in the module scope after +the class is defined. See :ref:`type-params` for a detailed description of the +runtime semantics of type parameters. + +In order to support these scoping semantics, a new kind of scope is introduced, +the :ref:`annotation scope `. Annotation scopes behave for the +most part like function scopes, but interact differently with enclosing class scopes. +In Python 3.13, :term:`annotations ` will also be evaluated in +annotation scopes. + +See :pep:`695` for more details. + +(PEP written by Eric Traut. Implementation by Jelle Zijlstra, Eric Traut, +and others in :gh:`103764`.) .. _whatsnew312-pep701: @@ -244,52 +331,6 @@ are parsed with the PEG parser, error messages can be more precise and show the Maureira-Fredes and Marta Gómez in :gh:`102856`. PEP written by Pablo Galindo, Batuhan Taskaya, Lysandros Nikolaou and Marta Gómez). -.. _whatsnew312-pep709: - -PEP 709: Comprehension inlining -------------------------------- - -Dictionary, list, and set comprehensions are now inlined, rather than creating a -new single-use function object for each execution of the comprehension. This -speeds up execution of a comprehension by up to two times. -See :pep:`709` for further details. - -Comprehension iteration variables remain isolated and don't overwrite a -variable of the same name in the outer scope, nor are they visible after the -comprehension. Inlining does result in a few visible behavior changes: - -* There is no longer a separate frame for the comprehension in tracebacks, - and tracing/profiling no longer shows the comprehension as a function call. -* The :mod:`symtable` module will no longer produce child symbol tables for each - comprehension; instead, the comprehension's locals will be included in the - parent function's symbol table. -* Calling :func:`locals` inside a comprehension now includes variables - from outside the comprehension, and no longer includes the synthetic ``.0`` - variable for the comprehension "argument". -* A comprehension iterating directly over ``locals()`` (e.g. ``[k for k in - locals()]``) may see "RuntimeError: dictionary changed size during iteration" - when run under tracing (e.g. code coverage measurement). This is the same - behavior already seen in e.g. ``for k in locals():``. To avoid the error, first - create a list of keys to iterate over: ``keys = list(locals()); [k for k in - keys]``. - -(Contributed by Carl Meyer and Vladimir Matveev in :pep:`709`.) - -.. _whatsnew312-pep688: - -PEP 688: Making the buffer protocol accessible in Python --------------------------------------------------------- - -:pep:`688` introduces a way to use the :ref:`buffer protocol ` -from Python code. Classes that implement the :meth:`~object.__buffer__` method -are now usable as buffer types. - -The new :class:`collections.abc.Buffer` ABC provides a standard -way to represent buffer objects, for example in type annotations. -The new :class:`inspect.BufferFlags` enum represents the flags that -can be used to customize buffer creation. -(Contributed by Jelle Zijlstra in :gh:`102500`.) - .. _whatsnew312-pep684: PEP 684: A Per-Interpreter GIL @@ -333,7 +374,105 @@ This means that you only pay for what you use, providing support for near-zero overhead debuggers and coverage tools. See :mod:`sys.monitoring` for details. -(Contributed by Mark Shannon in :gh:`103083`.) +(Contributed by Mark Shannon in :gh:`103082`.) + +.. _whatsnew312-pep688: + +PEP 688: Making the buffer protocol accessible in Python +-------------------------------------------------------- + +:pep:`688` introduces a way to use the :ref:`buffer protocol ` +from Python code. Classes that implement the :meth:`~object.__buffer__` method +are now usable as buffer types. + +The new :class:`collections.abc.Buffer` ABC provides a standard +way to represent buffer objects, for example in type annotations. +The new :class:`inspect.BufferFlags` enum represents the flags that +can be used to customize buffer creation. +(Contributed by Jelle Zijlstra in :gh:`102500`.) + +.. _whatsnew312-pep709: + +PEP 709: Comprehension inlining +------------------------------- + +Dictionary, list, and set comprehensions are now inlined, rather than creating a +new single-use function object for each execution of the comprehension. This +speeds up execution of a comprehension by up to two times. +See :pep:`709` for further details. + +Comprehension iteration variables remain isolated and don't overwrite a +variable of the same name in the outer scope, nor are they visible after the +comprehension. Inlining does result in a few visible behavior changes: + +* There is no longer a separate frame for the comprehension in tracebacks, + and tracing/profiling no longer shows the comprehension as a function call. +* The :mod:`symtable` module will no longer produce child symbol tables for each + comprehension; instead, the comprehension's locals will be included in the + parent function's symbol table. +* Calling :func:`locals` inside a comprehension now includes variables + from outside the comprehension, and no longer includes the synthetic ``.0`` + variable for the comprehension "argument". +* A comprehension iterating directly over ``locals()`` (e.g. ``[k for k in + locals()]``) may see "RuntimeError: dictionary changed size during iteration" + when run under tracing (e.g. code coverage measurement). This is the same + behavior already seen in e.g. ``for k in locals():``. To avoid the error, first + create a list of keys to iterate over: ``keys = list(locals()); [k for k in + keys]``. + +(Contributed by Carl Meyer and Vladimir Matveev in :pep:`709`.) + +Improved Error Messages +----------------------- + +* Modules from the standard library are now potentially suggested as part of + the error messages displayed by the interpreter when a :exc:`NameError` is + raised to the top level. (Contributed by Pablo Galindo in :gh:`98254`.) + + >>> sys.version_info + Traceback (most recent call last): + File "", line 1, in + NameError: name 'sys' is not defined. Did you forget to import 'sys'? + +* Improve the error suggestion for :exc:`NameError` exceptions for instances. + Now if a :exc:`NameError` is raised in a method and the instance has an + attribute that's exactly equal to the name in the exception, the suggestion + will include ``self.`` instead of the closest match in the method + scope. (Contributed by Pablo Galindo in :gh:`99139`.) + + >>> class A: + ... def __init__(self): + ... self.blech = 1 + ... + ... def foo(self): + ... somethin = blech + ... + >>> A().foo() + Traceback (most recent call last): + File "", line 1 + somethin = blech + ^^^^^ + NameError: name 'blech' is not defined. Did you mean: 'self.blech'? + +* Improve the :exc:`SyntaxError` error message when the user types ``import x + from y`` instead of ``from y import x``. (Contributed by Pablo Galindo in :gh:`98931`.) + + >>> import a.y.z from b.y.z + Traceback (most recent call last): + File "", line 1 + import a.y.z from b.y.z + ^^^^^^^^^^^^^^^^^^^^^^^ + SyntaxError: Did you mean to use 'from ... import ...' instead? + +* :exc:`ImportError` exceptions raised from failed ``from import + `` statements now include suggestions for the value of ```` based on the + available names in ````. (Contributed by Pablo Galindo in :gh:`91058`.) + + >>> from collections import chainmap + Traceback (most recent call last): + File "", line 1, in + ImportError: cannot import name 'chainmap' from 'collections'. Did you mean: 'ChainMap'? + New Features Related to Type Hints ================================== @@ -398,70 +537,6 @@ See :pep:`698` for more details. (Contributed by Steven Troxler in :gh:`101561`.) -.. _whatsnew312-pep695: - -PEP 695: Type Parameter Syntax ------------------------------- - -Generic classes and functions under :pep:`484` were declared using a verbose syntax -that left the scope of type parameters unclear and required explicit declarations of -variance. - -:pep:`695` introduces a new, more compact and explicit way to create -:ref:`generic classes ` and :ref:`functions `:: - - def max[T](args: Iterable[T]) -> T: - ... - - class list[T]: - def __getitem__(self, index: int, /) -> T: - ... - - def append(self, element: T) -> None: - ... - -In addition, the PEP introduces a new way to declare :ref:`type aliases ` -using the :keyword:`type` statement, which creates an instance of -:class:`~typing.TypeAliasType`:: - - type Point = tuple[float, float] - -Type aliases can also be :ref:`generic `:: - - type Point[T] = tuple[T, T] - -The new syntax allows declaring :class:`~typing.TypeVarTuple` -and :class:`~typing.ParamSpec` parameters, as well as :class:`~typing.TypeVar` -parameters with bounds or constraints:: - - type IntFunc[**P] = Callable[P, int] # ParamSpec - type LabeledTuple[*Ts] = tuple[str, *Ts] # TypeVarTuple - type HashableSequence[T: Hashable] = Sequence[T] # TypeVar with bound - type IntOrStrSequence[T: (int, str)] = Sequence[T] # TypeVar with constraints - -The value of type aliases and the bound and constraints of type variables -created through this syntax are evaluated only on demand (see -:ref:`lazy evaluation `). This means type aliases are able to -refer to other types defined later in the file. - -Type parameters declared through a type parameter list are visible within the -scope of the declaration and any nested scopes, but not in the outer scope. For -example, they can be used in the type annotations for the methods of a generic -class or in the class body. However, they cannot be used in the module scope after -the class is defined. See :ref:`type-params` for a detailed description of the -runtime semantics of type parameters. - -In order to support these scoping semantics, a new kind of scope is introduced, -the :ref:`annotation scope `. Annotation scopes behave for the -most part like function scopes, but interact differently with enclosing class scopes. -In Python 3.13, :term:`annotations ` will also be evaluated in -annotation scopes. - -See :pep:`695` for more details. - -(PEP written by Eric Traut. Implementation by Jelle Zijlstra, Eric Traut, -and others in :gh:`103764`.) - Other Language Changes ====================== @@ -1020,6 +1095,13 @@ CPython bytecode changes * Add the :opcode:`LOAD_SUPER_ATTR` instruction. (Contributed by Carl Meyer and Vladimir Matveev in :gh:`103497`.) +FOR_ITER new behavior is not mentioned +The fact that POP_JUMP_IF_* family of instructions are now real instructions is not mentioned +YIELD_VALUE need for an argument is not mentioned + + + + Demos and Tools =============== @@ -1578,6 +1660,8 @@ unittest * Remove many long-deprecated :mod:`unittest` features: + .. _unittest-TestCase-removed-aliases: + * A number of :class:`~unittest.TestCase` method aliases: ============================ =============================== =============== @@ -1811,6 +1895,7 @@ C API Changes New Features ------------ +.. _whatsnew312-pep697: * :pep:`697`: Introduce the :ref:`Unstable C API tier `, intended for low-level tools like debuggers and JIT compilers. @@ -1938,6 +2023,8 @@ New Features to replace the legacy-api :c:func:`!PyErr_Display`. (Contributed by Irit Katriel in :gh:`102755`). +.. _whatsnew312-pep683: + * :pep:`683`: Introduce *Immortal Objects*, which allows objects to bypass reference counts, and related changes to the C-API: