Merge remote-tracking branch 'origin/main' into fix-tg

Doc/c-api/long.rst

@@ -113,24 +113,28 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
  retrieved from the resulting value using :c:func:`PyLong_AsVoidPtr`.
 
 
-.. c:function:: PyObject* PyLong_FromNativeBytes(const void* buffer, size_t n_bytes, int endianness)
+.. c:function:: PyObject* PyLong_FromNativeBytes(const void* buffer, size_t n_bytes, int flags)
 
  Create a Python integer from the value contained in the first *n_bytes* of
  *buffer*, interpreted as a two's-complement signed number.
 
- *endianness* may be passed ``-1`` for the native endian that CPython was
- compiled with, or else ``0`` for big endian and ``1`` for little.
+ *flags* are as for :c:func:`PyLong_AsNativeBytes`. Passing ``-1`` will select
+ the native endian that CPython was compiled with and assume that the
+ most-significant bit is a sign bit. Passing
+ ``Py_ASNATIVEBYTES_UNSIGNED_BUFFER`` will produce the same result as calling
+ :c:func:`PyLong_FromUnsignedNativeBytes`. Other flags are ignored.
 
  .. versionadded:: 3.13
 
 
-.. c:function:: PyObject* PyLong_FromUnsignedNativeBytes(const void* buffer, size_t n_bytes, int endianness)
+.. c:function:: PyObject* PyLong_FromUnsignedNativeBytes(const void* buffer, size_t n_bytes, int flags)
 
  Create a Python integer from the value contained in the first *n_bytes* of
  *buffer*, interpreted as an unsigned number.
 
- *endianness* may be passed ``-1`` for the native endian that CPython was
- compiled with, or else ``0`` for big endian and ``1`` for little.
+ *flags* are as for :c:func:`PyLong_AsNativeBytes`. Passing ``-1`` will select
+ the native endian that CPython was compiled with and assume that the
+ most-significant bit is not a sign bit. Flags other than endian are ignored.
 
  .. versionadded:: 3.13
 
@@ -354,14 +358,41 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
  Returns ``NULL`` on error. Use :c:func:`PyErr_Occurred` to disambiguate.
 
 
-.. c:function:: Py_ssize_t PyLong_AsNativeBytes(PyObject *pylong, void* buffer, Py_ssize_t n_bytes, int endianness)
+.. c:function:: Py_ssize_t PyLong_AsNativeBytes(PyObject *pylong, void* buffer, Py_ssize_t n_bytes, int flags)
 
- Copy the Python integer value to a native *buffer* of size *n_bytes*::
+ Copy the Python integer value *pylong* to a native *buffer* of size
+ *n_bytes*. The *flags* can be set to ``-1`` to behave similarly to a C cast,
+ or to values documented below to control the behavior.
+
+ Returns ``-1`` with an exception raised on error. This may happen if
+ *pylong* cannot be interpreted as an integer, or if *pylong* was negative
+ and the ``Py_ASNATIVEBYTES_REJECT_NEGATIVE`` flag was set.
+
+ Otherwise, returns the number of bytes required to store the value.
+ If this is equal to or less than *n_bytes*, the entire value was copied.
+ All *n_bytes* of the buffer are written: large buffers are padded with
+ zeroes.
+
+ If the returned value is greater than than *n_bytes*, the value was
+ truncated: as many of the lowest bits of the value as could fit are written,
+ and the higher bits are ignored. This matches the typical behavior
+ of a C-style downcast.
+
+ .. note::
+
+ Overflow is not considered an error. If the returned value
+ is larger than *n_bytes*, most significant bits were discarded.
+
+ ``0`` will never be returned.
+
+ Values are always copied as two's-complement.
+
+ Usage example::
 
  int32_t value;
  Py_ssize_t bytes = PyLong_AsNativeBits(pylong, &value, sizeof(value), -1);
  if (bytes < 0) {
- // A Python exception was set with the reason.
+ // Failed. A Python exception was set with the reason.
  return NULL;
  }
  else if (bytes <= (Py_ssize_t)sizeof(value)) {
@@ -372,19 +403,24 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
  // lowest bits of pylong.
  }
 
- The above example may look *similar* to
- :c:func:`PyLong_As* <PyLong_AsSize_t>`
- but instead fills in a specific caller defined type and never raises an
- error about of the :class:`int` *pylong*'s value regardless of *n_bytes*
- or the returned byte count.
+ Passing zero to *n_bytes* will return the size of a buffer that would
+ be large enough to hold the value. This may be larger than technically
+ necessary, but not unreasonably so.
 
- To get at the entire potentially big Python value, this can be used to
- reserve enough space and copy it::
+ .. note::
+
+ Passing *n_bytes=0* to this function is not an accurate way to determine
+ the bit length of a value.
+
+ If *n_bytes=0*, *buffer* may be ``NULL``.
+
+ To get at the entire Python value of an unknown size, the function can be
+ called twice: first to determine the buffer size, then to fill it::
 
  // Ask how much space we need.
  Py_ssize_t expected = PyLong_AsNativeBits(pylong, NULL, 0, -1);
  if (expected < 0) {
- // A Python exception was set with the reason.
+ // Failed. A Python exception was set with the reason.
  return NULL;
  }
  assert(expected != 0); // Impossible per the API definition.
@@ -395,11 +431,11 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
  }
  // Safely get the entire value.
  Py_ssize_t bytes = PyLong_AsNativeBits(pylong, bignum, expected, -1);
- if (bytes < 0) { // Exception set.
+ if (bytes < 0) { // Exception has been set.
  free(bignum);
  return NULL;
  }
- else if (bytes > expected) { // Be safe, should not be possible.
+ else if (bytes > expected) { // This should not be possible.
  PyErr_SetString(PyExc_RuntimeError,
  "Unexpected bignum truncation after a size check.");
  free(bignum);
@@ -409,35 +445,51 @@ distinguished from a number. Use :c:func:`PyErr_Occurred` to disambiguate.
  // ... use bignum ...
  free(bignum);
 
- *endianness* may be passed ``-1`` for the native endian that CPython was
- compiled with, or ``0`` for big endian and ``1`` for little.
-
- Returns ``-1`` with an exception raised if *pylong* cannot be interpreted as
- an integer. Otherwise, return the size of the buffer required to store the
- value. If this is equal to or less than *n_bytes*, the entire value was
- copied. ``0`` will never be returned.
-
- Unless an exception is raised, all *n_bytes* of the buffer will always be
- written. In the case of truncation, as many of the lowest bits of the value
- as could fit are written. This allows the caller to ignore all non-negative
- results if the intent is to match the typical behavior of a C-style
- downcast. No exception is set on truncation.
-
- Values are always copied as two's-complement and sufficient buffer will be
- requested to include a sign bit. For example, this may cause an value that
- fits into 8 bytes when treated as unsigned to request 9 bytes, even though
- all eight bytes were copied into the buffer. What has been omitted is the
- zero sign bit -- redundant if the caller's intention is to treat the value
- as unsigned.
-
- Passing zero to *n_bytes* will return the size of a buffer that would
- be large enough to hold the value. This may be larger than technically
- necessary, but not unreasonably so.
+ *flags* is either ``-1`` (``Py_ASNATIVEBYTES_DEFAULTS``) to select defaults
+ that behave most like a C cast, or a combintation of the other flags in
+ the table below.
+ Note that ``-1`` cannot be combined with other flags.
+
+ Currently, ``-1`` corresponds to
+ ``Py_ASNATIVEBYTES_NATIVE_ENDIAN | Py_ASNATIVEBYTES_UNSIGNED_BUFFER``.
+
+ ============================================= ======
+ Flag Value
+ ============================================= ======
+ .. c:macro:: Py_ASNATIVEBYTES_DEFAULTS ``-1``
+ .. c:macro:: Py_ASNATIVEBYTES_BIG_ENDIAN ``0``
+ .. c:macro:: Py_ASNATIVEBYTES_LITTLE_ENDIAN ``1``
+ .. c:macro:: Py_ASNATIVEBYTES_NATIVE_ENDIAN ``3``
+ .. c:macro:: Py_ASNATIVEBYTES_UNSIGNED_BUFFER ``4``
+ .. c:macro:: Py_ASNATIVEBYTES_REJECT_NEGATIVE ``8``
+ ============================================= ======
+
+ Specifying ``Py_ASNATIVEBYTES_NATIVE_ENDIAN`` will override any other endian
+ flags. Passing ``2`` is reserved.
+
+ By default, sufficient buffer will be requested to include a sign bit.
+ For example, when converting 128 with *n_bytes=1*, the function will return
+ 2 (or more) in order to store a zero sign bit.
+
+ If ``Py_ASNATIVEBYTES_UNSIGNED_BUFFER`` is specified, a zero sign bit
+ will be omitted from size calculations. This allows, for example, 128 to fit
+ in a single-byte buffer. If the destination buffer is later treated as
+ signed, a positive input value may become negative.
+ Note that the flag does not affect handling of negative values: for those,
+ space for a sign bit is always requested.
+
+ Specifying ``Py_ASNATIVEBYTES_REJECT_NEGATIVE`` causes an exception to be set
+ if *pylong* is negative. Without this flag, negative values will be copied
+ provided there is enough space for at least one sign bit, regardless of
+ whether ``Py_ASNATIVEBYTES_UNSIGNED_BUFFER`` was specified.
 
  .. note::
 
- Passing *n_bytes=0* to this function is not an accurate way to determine
- the bit length of a value.
+ With the default *flags* (``-1``, or *UNSIGNED_BUFFER* without
+ *REJECT_NEGATIVE*), multiple Python integers can map to a single value
+ without overflow. For example, both ``255`` and ``-1`` fit a single-byte
+ buffer and set all its bits.
+ This matches typical C cast behavior.
 
  .. versionadded:: 3.13
 

Doc/library/importlib.resources.rst

@@ -97,3 +97,181 @@ for example, a package and its resources can be imported from a zip file using
 
  .. versionchanged:: 3.12
  Added support for *traversable* representing a directory.
+
+
+.. _importlib_resources_functional:
+
+Functional API
+^^^^^^^^^^^^^^
+
+A set of simplified, backwards-compatible helpers is available.
+These allow common operations in a single function call.
+
+For all the following functions:
+
+- *anchor* is an :class:`~importlib.resources.Anchor`,
+ as in :func:`~importlib.resources.files`.
+ Unlike in ``files``, it may not be omitted.
+
+- *path_names* are components of a resource's path name, relative to
+ the anchor.
+ For example, to get the text of resource named ``info.txt``, use::
+
+ importlib.resources.read_text(my_module, "info.txt")
+
+ Like :meth:`Traversable.joinpath <importlib.resources.abc.Traversable>`,
+ The individual components should use forward slashes (``/``)
+ as path separators.
+ For example, the following are equivalent::
+
+ importlib.resources.read_binary(my_module, "pics/painting.png")
+ importlib.resources.read_binary(my_module, "pics", "painting.png")
+
+ For backward compatibility reasons, functions that read text require
+ an explicit *encoding* argument if multiple *path_names* are given.
+ For example, to get the text of ``info/chapter1.txt``, use::
+
+ importlib.resources.read_text(my_module, "info", "chapter1.txt",
+ encoding='utf-8')
+
+.. function:: open_binary(anchor, *path_names)
+
+ Open the named resource for binary reading.
+
+ See :ref:`the introduction <importlib_resources_functional>` for
+ details on *anchor* and *path_names*.
+
+ This function returns a :class:`~typing.BinaryIO` object,
+ that is, a binary stream open for reading.
+
+ This function is roughly equivalent to::
+
+ files(anchor).joinpath(*path_names).open('rb')
+
+ .. versionchanged:: 3.13
+ Multiple *path_names* are accepted.
+
+
+.. function:: open_text(anchor, *path_names, encoding='utf-8', errors='strict')
+
+ Open the named resource for text reading.
+ By default, the contents are read as strict UTF-8.
+
+ See :ref:`the introduction <importlib_resources_functional>` for
+ details on *anchor* and *path_names*.
+ *encoding* and *errors* have the same meaning as in built-in :func:`open`.
+
+ For backward compatibility reasons, the *encoding* argument must be given
+ explicitly if there are multiple *path_names*.
+ This limitation is scheduled to be removed in Python 3.15.
+
+ This function returns a :class:`~typing.TextIO` object,
+ that is, a text stream open for reading.
+
+ This function is roughly equivalent to::
+
+ files(anchor).joinpath(*path_names).open('r', encoding=encoding)
+
+ .. versionchanged:: 3.13
+ Multiple *path_names* are accepted.
+ *encoding* and *errors* must be given as keyword arguments.
+
+
+.. function:: read_binary(anchor, *path_names)
+
+ Read and return the contents of the named resource as :class:`bytes`.
+
+ See :ref:`the introduction <importlib_resources_functional>` for
+ details on *anchor* and *path_names*.
+
+ This function is roughly equivalent to::
+
+ files(anchor).joinpath(*path_names).read_bytes()
+
+ .. versionchanged:: 3.13
+ Multiple *path_names* are accepted.
+
+
+.. function:: read_text(anchor, *path_names, encoding='utf-8', errors='strict')
+
+ Read and return the contents of the named resource as :class:`str`.
+ By default, the contents are read as strict UTF-8.
+
+ See :ref:`the introduction <importlib_resources_functional>` for
+ details on *anchor* and *path_names*.
+ *encoding* and *errors* have the same meaning as in built-in :func:`open`.
+
+ For backward compatibility reasons, the *encoding* argument must be given
+ explicitly if there are multiple *path_names*.
+ This limitation is scheduled to be removed in Python 3.15.
+
+ This function is roughly equivalent to::
+
+ files(anchor).joinpath(*path_names).read_text(encoding=encoding)
+
+ .. versionchanged:: 3.13
+ Multiple *path_names* are accepted.
+ *encoding* and *errors* must be given as keyword arguments.
+
+
+.. function:: path(anchor, *path_names)
+
+ Provides the path to the *resource* as an actual file system path. This
+ function returns a context manager for use in a :keyword:`with` statement.
+ The context manager provides a :class:`pathlib.Path` object.
+
+ Exiting the context manager cleans up any temporary files created, e.g.
+ when the resource needs to be extracted from a zip file.
+
+ For example, the :meth:`~pathlib.Path.stat` method requires
+ an actual file system path; it can be used like this::
+
+ with importlib.resources.path(anchor, "resource.txt") as fspath:
+ result = fspath.stat()
+
+ See :ref:`the introduction <importlib_resources_functional>` for
+ details on *anchor* and *path_names*.
+
+ This function is roughly equivalent to::
+
+ as_file(files(anchor).joinpath(*path_names))
+
+ .. versionchanged:: 3.13
+ Multiple *path_names* are accepted.
+ *encoding* and *errors* must be given as keyword arguments.
+
+
+.. function:: is_resource(anchor, *path_names)
+
+ Return ``True`` if the named resource exists, otherwise ``False``.
+ This function does not consider directories to be resources.
+
+ See :ref:`the introduction <importlib_resources_functional>` for
+ details on *anchor* and *path_names*.
+
+ This function is roughly equivalent to::
+
+ files(anchor).joinpath(*path_names).is_file()
+
+ .. versionchanged:: 3.13
+ Multiple *path_names* are accepted.
+
+
+.. function:: contents(anchor, *path_names)
+
+ Return an iterable over the named items within the package or path.
+ The iterable returns names of resources (e.g. files) and non-resources
+ (e.g. directories) as :class:`str`.
+ The iterable does not recurse into subdirectories.
+
+ See :ref:`the introduction <importlib_resources_functional>` for
+ details on *anchor* and *path_names*.
+
+ This function is roughly equivalent to::
+
+ for resource in files(anchor).joinpath(*path_names).iterdir():
+ yield resource.name
+
+ .. deprecated:: 3.11
+ Prefer ``iterdir()`` as above, which offers more control over the
+ results and richer functionality.