Merge branch 'main' into replace_pylong_numbits

.github/workflows/jit.yml

@@ -150,3 +150,21 @@ jobs:
  ./configure --enable-experimental-jit ${{ matrix.debug && '--with-pydebug' || '--enable-optimizations ' }} --build=x86_64-linux-gnu --host="$HOST" --with-build-python=../build/bin/python3 --with-pkg-config=no ac_cv_buggy_getaddrinfo=no ac_cv_file__dev_ptc=no ac_cv_file__dev_ptmx=yes
  make all --jobs 4
  ./python -m test --ignorefile=Tools/jit/ignore-tests-emulated-linux.txt --multiprocess 0 --timeout 4500 --verbose2 --verbose3
+
+ jit-with-disabled-gil:
+ name: Free-Threaded (Debug)
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with:
+ python-version: '3.11'
+ - name: Build with JIT enabled and GIL disabled
+ run: |
+ sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh 18
+ export PATH="$(llvm-config-18 --bindir):$PATH"
+ ./configure --enable-experimental-jit --with-pydebug --disable-gil
+ make all --jobs 4
+ - name: Run tests
+ run: |
+ ./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3

.github/workflows/mypy.yml

@@ -34,6 +34,7 @@ concurrency:
 jobs:
  mypy:
  strategy:
+ fail-fast: false
  matrix:
  target: [
  "Lib/_pyrepl",

.github/workflows/reusable-docs.yml

@@ -62,7 +62,8 @@ jobs:
  python Doc/tools/check-warnings.py \
  --annotate-diff '${{ env.branch_base }}' '${{ env.branch_pr }}' \
  --fail-if-regression \
- --fail-if-improved
+ --fail-if-improved \
+ --fail-if-new-news-nit
 
  # This build doesn't use problem matchers or check annotations
  build_doc_oldest_supported_sphinx:

Doc/c-api/frame.rst

@@ -121,17 +121,18 @@ See also :ref:`Reflection <reflection>`.
 .. c:function:: PyObject* PyFrame_GetLocals(PyFrameObject *frame)
 
  Get the *frame*'s :attr:`~frame.f_locals` attribute.
- If the frame refers to a function or comprehension, this returns
- a write-through proxy object that allows modifying the locals.
- In all other cases (classes, modules) it returns the :class:`dict`
- representing the frame locals directly.
+ If the frame refers to an :term:`optimized scope`, this returns a
+ write-through proxy object that allows modifying the locals.
+ In all other cases (classes, modules, :func:`exec`, :func:`eval`) it returns
+ the mapping representing the frame locals directly (as described for
+ :func:`locals`).
 
  Return a :term:`strong reference`.
 
  .. versionadded:: 3.11
 
  .. versionchanged:: 3.13
- Return a proxy object for functions and comprehensions.
+ As part of :pep:`667`, return a proxy object for optimized scopes.
 
 
 .. c:function:: int PyFrame_GetLineNumber(PyFrameObject *frame)

Doc/c-api/hash.rst

@@ -29,6 +29,12 @@ See also the :c:member:`PyTypeObject.tp_hash` member and :ref:`numeric-hash`.
 
  .. versionadded:: 3.13
 
+.. c:macro:: PyHASH_MULTIPLIER
+
+ Prime multiplier used in string and various other hashes.
+
+ .. versionadded:: 3.13
+
 .. c:macro:: PyHASH_INF
 
  The hash value returned for a positive infinity.

Doc/c-api/module.rst

@@ -427,14 +427,14 @@ The available slot types are:
  This slot is ignored by Python builds not configured with
  :option:`--disable-gil`. Otherwise, it determines whether or not importing
  this module will cause the GIL to be automatically enabled. See
- :envvar:`PYTHON_GIL` and :option:`-X gil <-X>` for more detail.
+ :ref:`free-threaded-cpython` for more detail.
 
  Multiple ``Py_mod_gil`` slots may not be specified in one module definition.
 
  If ``Py_mod_gil`` is not specified, the import machinery defaults to
  ``Py_MOD_GIL_USED``.
 
- .. versionadded: 3.13
+ .. versionadded:: 3.13
 
 See :PEP:`489` for more details on multi-phase initialization.
 

Doc/c-api/monitoring.rst

@@ -121,10 +121,10 @@ See :mod:`sys.monitoring` for descriptions of the events.
  :c:func:`PyErr_GetRaisedException`).
 
 
-.. c:function:: int PyMonitoring_FireStopIterationEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset)
+.. c:function:: int PyMonitoring_FireStopIterationEvent(PyMonitoringState *state, PyObject *codelike, int32_t offset, PyObject *value)
 
- Fire a ``STOP_ITERATION`` event with the current exception (as returned by
- :c:func:`PyErr_GetRaisedException`).
+ Fire a ``STOP_ITERATION`` event. If ``value`` is an instance of :exc:`StopIteration`, it is used. Otherwise,
+ a new :exc:`StopIteration` instance is created with ``value`` as its argument.
 
 
 Managing the Monitoring State

Doc/glossary.rst

@@ -889,6 +889,15 @@ Glossary
  (methods). Also the ultimate base class of any :term:`new-style
  class`.
 
+ optimized scope
+ A scope where target local variable names are reliably known to the
+ compiler when the code is compiled, allowing optimization of read and
+ write access to these names. The local namespaces for functions,
+ generators, coroutines, comprehensions, and generator expressions are
+ optimized in this fashion. Note: most interpreter optimizations are
+ applied to all scopes, only those relying on a known set of local
+ and nonlocal variable names are restricted to optimized scopes.
+
  package
  A Python :term:`module` which can contain submodules or recursively,
  subpackages. Technically, a package is a Python module with a

Doc/library/ast.rst

@@ -2472,6 +2472,20 @@ effects on the compilation of a program:
  .. versionadded:: 3.8
 
 
+.. function:: compare(a, b, /, *, compare_attributes=False)
+
+ Recursively compares two ASTs.
+
+ *compare_attributes* affects whether AST attributes are considered
+ in the comparison. If *compare_attributes* is ``False`` (default), then
+ attributes are ignored. Otherwise they must all be equal. This
+ option is useful to check whether the ASTs are structurally equal but
+ differ in whitespace or similar details. Attributes include line numbers
+ and column offsets.
+
+ .. versionadded:: 3.14
+
+
 .. _ast-cli:
 
 Command-Line Usage

Doc/library/code.rst

@@ -18,9 +18,9 @@ build applications which provide an interactive interpreter prompt.
  This class deals with parsing and interpreter state (the user's namespace); it
  does not deal with input buffering or prompting or input file naming (the
  filename is always passed in explicitly). The optional *locals* argument
- specifies the dictionary in which code will be executed; it defaults to a newly
- created dictionary with key ``'__name__'`` set to ``'__console__'`` and key
- ``'__doc__'`` set to ``None``.
+ specifies a mapping to use as the namespace in which code will be executed;
+ it defaults to a newly created dictionary with key ``'__name__'`` set to
+ ``'__console__'`` and key ``'__doc__'`` set to ``None``.
 
 
 .. class:: InteractiveConsole(locals=None, filename="<console>", local_exit=False)

Doc/library/dataclasses.rst

@@ -185,7 +185,10 @@ Module contents
  - *slots*: If true (the default is ``False``), :attr:`~object.__slots__` attribute
  will be generated and new class will be returned instead of the original one.
  If :attr:`!__slots__` is already defined in the class, then :exc:`TypeError`
- is raised.
+ is raised. Calling no-arg :func:`super` in dataclasses using ``slots=True`` will result in
+ the following exception being raised:
+ ``TypeError: super(type, obj): obj must be an instance or subtype of type``.
+ The two-arg :func:`super` is a valid workaround. See :gh:`90562` for full details.
 
  .. versionadded:: 3.10
 
@@ -616,7 +619,8 @@ methods will raise a :exc:`FrozenInstanceError` when invoked.
 There is a tiny performance penalty when using ``frozen=True``:
 :meth:`~object.__init__` cannot use simple assignment to initialize fields, and
 must use :meth:`!object.__setattr__`.
-.. Make sure to not remove "object" from "object.__setattr__" in the above markup
+
+.. Make sure to not remove "object" from "object.__setattr__" in the above markup!
 
 .. _dataclasses-inheritance:
 

Doc/library/dis.rst

@@ -929,12 +929,13 @@ iterations of the loop.
  Exception representation on the stack now consist of one, not three, items.
 
 
-.. opcode:: LOAD_ASSERTION_ERROR
+.. opcode:: LOAD_COMMON_CONSTANT
 
- Pushes :exc:`AssertionError` onto the stack. Used by the :keyword:`assert`
- statement.
+ Pushes a common constant onto the stack. The interpreter contains a hardcoded
+ list of constants supported by this instruction. Used by the :keyword:`assert`
+ statement to load :exc:`AssertionError`.
 
- .. versionadded:: 3.9
+ .. versionadded:: 3.14
 
 
 .. opcode:: LOAD_BUILD_CLASS

Doc/library/functions.rst

@@ -543,18 +543,19 @@ are always available. They are listed here in alphabetical order.
 
  The *expression* argument is parsed and evaluated as a Python expression
  (technically speaking, a condition list) using the *globals* and *locals*
- dictionaries as global and local namespace. If the *globals* dictionary is
+ mappings as global and local namespace. If the *globals* dictionary is
  present and does not contain a value for the key ``__builtins__``, a
  reference to the dictionary of the built-in module :mod:`builtins` is
  inserted under that key before *expression* is parsed. That way you can
  control what builtins are available to the executed code by inserting your
  own ``__builtins__`` dictionary into *globals* before passing it to
- :func:`eval`. If the *locals* dictionary is omitted it defaults to the
- *globals* dictionary. If both dictionaries are omitted, the expression is
+ :func:`eval`. If the *locals* mapping is omitted it defaults to the
+ *globals* dictionary. If both mappings are omitted, the expression is
  executed with the *globals* and *locals* in the environment where
- :func:`eval` is called. Note, *eval()* does not have access to the
+ :func:`eval` is called. Note, *eval()* will only have access to the
  :term:`nested scopes <nested scope>` (non-locals) in the enclosing
- environment.
+ environment if they are already referenced in the scope that is calling
+ :func:`eval` (e.g. via a :keyword:`nonlocal` statement).
 
  Example:
 
@@ -587,6 +588,11 @@ are always available. They are listed here in alphabetical order.
 
  The *globals* and *locals* arguments can now be passed as keywords.
 
+ .. versionchanged:: 3.13
+
+ The semantics of the default *locals* namespace have been adjusted as
+ described for the :func:`locals` builtin.
+
 .. index:: pair: built-in function; exec
 
 .. function:: exec(source, /, globals=None, locals=None, *, closure=None)
@@ -608,9 +614,15 @@ are always available. They are listed here in alphabetical order.
  will be used for both the global and the local variables. If *globals* and
  *locals* are given, they are used for the global and local variables,
  respectively. If provided, *locals* can be any mapping object. Remember
- that at the module level, globals and locals are the same dictionary. If exec
- gets two separate objects as *globals* and *locals*, the code will be
- executed as if it were embedded in a class definition.
+ that at the module level, globals and locals are the same dictionary.
+
+ .. note::
+
+ When ``exec`` gets two separate objects as *globals* and *locals*, the
+ code will be executed as if it were embedded in a class definition. This
+ means functions and classes defined in the executed code will not be able
+ to access variables assigned at the top level (as the "top level"
+ variables are treated as class variables in a class definition).
 
  If the *globals* dictionary does not contain a value for the key
  ``__builtins__``, a reference to the dictionary of the built-in module
@@ -631,7 +643,7 @@ are always available. They are listed here in alphabetical order.
  .. note::
 
  The built-in functions :func:`globals` and :func:`locals` return the current
- global and local dictionary, respectively, which may be useful to pass around
+ global and local namespace, respectively, which may be useful to pass around
  for use as the second and third argument to :func:`exec`.
 
  .. note::
@@ -647,6 +659,11 @@ are always available. They are listed here in alphabetical order.
 
  The *globals* and *locals* arguments can now be passed as keywords.
 
+ .. versionchanged:: 3.13
+
+ The semantics of the default *locals* namespace have been adjusted as
+ described for the :func:`locals` builtin.
+
 
 .. function:: filter(function, iterable)
 
@@ -1052,39 +1069,51 @@ are always available. They are listed here in alphabetical order.
  variable names as the keys, and their currently bound references as the
  values.
 
- At module scope, as well as when using ``exec()`` or ``eval()`` with a
- single namespace, this function returns the same namespace as ``globals()``.
+ At module scope, as well as when using :func:`exec` or :func:`eval` with
+ a single namespace, this function returns the same namespace as
+ :func:`globals`.
 
  At class scope, it returns the namespace that will be passed to the
  metaclass constructor.
 
  When using ``exec()`` or ``eval()`` with separate local and global
- namespaces, it returns the local namespace passed in to the function call.
+ arguments, it returns the local namespace passed in to the function call.
 
  In all of the above cases, each call to ``locals()`` in a given frame of
  execution will return the *same* mapping object. Changes made through
- the mapping object returned from ``locals()`` will be visible as bound,
- rebound, or deleted local variables, and binding, rebinding, or deleting
- local variables will immediately affect the contents of the returned mapping
- object.
-
- At function scope (including for generators and coroutines), each call to
- ``locals()`` instead returns a fresh dictionary containing the current
- bindings of the function's local variables and any nonlocal cell references.
- In this case, name binding changes made via the returned dict are *not*
- written back to the corresponding local variables or nonlocal cell
- references, and binding, rebinding, or deleting local variables and nonlocal
- cell references does *not* affect the contents of previously returned
- dictionaries.
+ the mapping object returned from ``locals()`` will be visible as assigned,
+ reassigned, or deleted local variables, and assigning, reassigning, or
+ deleting local variables will immediately affect the contents of the
+ returned mapping object.
+
+ In an :term:`optimized scope` (including functions, generators, and
+ coroutines), each call to ``locals()`` instead returns a fresh dictionary
+ containing the current bindings of the function's local variables and any
+ nonlocal cell references. In this case, name binding changes made via the
+ returned dict are *not* written back to the corresponding local variables
+ or nonlocal cell references, and assigning, reassigning, or deleting local
+ variables and nonlocal cell references does *not* affect the contents
+ of previously returned dictionaries.
+
+ Calling ``locals()`` as part of a comprehension in a function, generator, or
+ coroutine is equivalent to calling it in the containing scope, except that
+ the comprehension's initialised iteration variables will be included. In
+ other scopes, it behaves as if the comprehension were running as a nested
+ function.
+
+ Calling ``locals()`` as part of a generator expression is equivalent to
+ calling it in a nested generator function.
+
+ .. versionchanged:: 3.12
+ The behaviour of ``locals()`` in a comprehension has been updated as
+ described in :pep:`709`.
 
  .. versionchanged:: 3.13
- In previous versions, the semantics of mutating the mapping object
- returned from this function were formally undefined. In CPython
- specifically, the mapping returned at function scope could be
- implicitly refreshed by other operations, such as calling ``locals()``
- again. Obtaining the legacy CPython behaviour now requires explicit
- calls to update the initially returned dictionary with the results
- of subsequent calls to ``locals()``.
+ As part of :pep:`667`, the semantics of mutating the mapping objects
+ returned from this function are now defined. The behavior in
+ :term:`optimized scopes <optimized scope>` is now as described above.
+ Aside from being defined, the behaviour in other scopes remains
+ unchanged from previous versions.
 
 
 .. function:: map(function, iterable, *iterables)
@@ -1971,14 +2000,18 @@ are always available. They are listed here in alphabetical order.
  :attr:`~object.__dict__` attributes (for example, classes use a
  :class:`types.MappingProxyType` to prevent direct dictionary updates).
 
- Without an argument, :func:`vars` acts like :func:`locals`. Note, the
- locals dictionary is only useful for reads since updates to the locals
- dictionary are ignored.
+ Without an argument, :func:`vars` acts like :func:`locals`.
 
  A :exc:`TypeError` exception is raised if an object is specified but
  it doesn't have a :attr:`~object.__dict__` attribute (for example, if
  its class defines the :attr:`~object.__slots__` attribute).
 
+ .. versionchanged:: 3.13
+
+ The result of calling this function without an argument has been
+ updated as described for the :func:`locals` builtin.
+
+
 .. function:: zip(*iterables, strict=False)
 
  Iterate over several iterables in parallel, producing tuples with an item

Doc/library/hashlib.rst

@@ -328,7 +328,7 @@ include a `salt <https://en.wikipedia.org/wiki/Salt_%28cryptography%29>`_.
  your application, read *Appendix A.2.2* of NIST-SP-800-132_. The answers
  on the `stackexchange pbkdf2 iterations question`_ explain in detail.
 
- *dklen* is the length of the derived key. If *dklen* is ``None`` then the
+ *dklen* is the length of the derived key in bytes. If *dklen* is ``None`` then the
  digest size of the hash algorithm *hash_name* is used, e.g. 64 for SHA-512.
 
  >>> from hashlib import pbkdf2_hmac
@@ -357,7 +357,7 @@ include a `salt <https://en.wikipedia.org/wiki/Salt_%28cryptography%29>`_.
 
  *n* is the CPU/Memory cost factor, *r* the block size, *p* parallelization
  factor and *maxmem* limits memory (OpenSSL 1.1.0 defaults to 32 MiB).
- *dklen* is the length of the derived key.
+ *dklen* is the length of the derived key in bytes.
 
  .. versionadded:: 3.6