Skip to content

Commit 49f6ff7

Browse files
authored
gh-98712: Clarify "readonly bytes-like object" semantics in C arg-parsing docs (#98710)
1 parent 88d565f commit 49f6ff7

File tree

1 file changed

+35
-19
lines changed

1 file changed

+35
-19
lines changed

Doc/c-api/arg.rst

+35-19
Original file line numberDiff line numberDiff line change
@@ -34,24 +34,39 @@ These formats allow accessing an object as a contiguous chunk of memory.
3434
You don't have to provide raw storage for the returned unicode or bytes
3535
area.
3636

37-
In general, when a format sets a pointer to a buffer, the buffer is
38-
managed by the corresponding Python object, and the buffer shares
39-
the lifetime of this object. You won't have to release any memory yourself.
40-
The only exceptions are ``es``, ``es#``, ``et`` and ``et#``.
41-
42-
However, when a :c:type:`Py_buffer` structure gets filled, the underlying
43-
buffer is locked so that the caller can subsequently use the buffer even
44-
inside a :c:type:`Py_BEGIN_ALLOW_THREADS` block without the risk of mutable data
45-
being resized or destroyed. As a result, **you have to call**
46-
:c:func:`PyBuffer_Release` after you have finished processing the data (or
47-
in any early abort case).
48-
4937
Unless otherwise stated, buffers are not NUL-terminated.
5038

51-
Some formats require a read-only :term:`bytes-like object`, and set a
52-
pointer instead of a buffer structure. They work by checking that
53-
the object's :c:member:`PyBufferProcs.bf_releasebuffer` field is ``NULL``,
54-
which disallows mutable objects such as :class:`bytearray`.
39+
There are three ways strings and buffers can be converted to C:
40+
41+
* Formats such as ``y*`` and ``s*`` fill a :c:type:`Py_buffer` structure.
42+
This locks the underlying buffer so that the caller can subsequently use
43+
the buffer even inside a :c:type:`Py_BEGIN_ALLOW_THREADS`
44+
block without the risk of mutable data being resized or destroyed.
45+
As a result, **you have to call** :c:func:`PyBuffer_Release` after you have
46+
finished processing the data (or in any early abort case).
47+
48+
* The ``es``, ``es#``, ``et`` and ``et#`` formats allocate the result buffer.
49+
**You have to call** :c:func:`PyMem_Free` after you have finished
50+
processing the data (or in any early abort case).
51+
52+
* .. _c-arg-borrowed-buffer:
53+
54+
Other formats take a :class:`str` or a read-only :term:`bytes-like object`,
55+
such as :class:`bytes`, and provide a ``const char *`` pointer to
56+
its buffer.
57+
In this case the buffer is "borrowed": it is managed by the corresponding
58+
Python object, and shares the lifetime of this object.
59+
You won't have to release any memory yourself.
60+
61+
To ensure that the underlying buffer may be safely borrowed, the object's
62+
:c:member:`PyBufferProcs.bf_releasebuffer` field must be ``NULL``.
63+
This disallows common mutable objects such as :class:`bytearray`,
64+
but also some read-only objects such as :class:`memoryview` of
65+
:class:`bytes`.
66+
67+
Besides this ``bf_releasebuffer`` requirement, there is no check to verify
68+
whether the input object is immutable (e.g. whether it would honor a request
69+
for a writable buffer, or whether another thread can mutate the data).
5570

5671
.. note::
5772

@@ -89,7 +104,7 @@ which disallows mutable objects such as :class:`bytearray`.
89104
Unicode objects are converted to C strings using ``'utf-8'`` encoding.
90105

91106
``s#`` (:class:`str`, read-only :term:`bytes-like object`) [const char \*, :c:type:`Py_ssize_t`]
92-
Like ``s*``, except that it doesn't accept mutable objects.
107+
Like ``s*``, except that it provides a :ref:`borrowed buffer <c-arg-borrowed-buffer>`.
93108
The result is stored into two C variables,
94109
the first one a pointer to a C string, the second one its length.
95110
The string may contain embedded null bytes. Unicode objects are converted
@@ -108,8 +123,9 @@ which disallows mutable objects such as :class:`bytearray`.
108123
pointer is set to ``NULL``.
109124

110125
``y`` (read-only :term:`bytes-like object`) [const char \*]
111-
This format converts a bytes-like object to a C pointer to a character
112-
string; it does not accept Unicode objects. The bytes buffer must not
126+
This format converts a bytes-like object to a C pointer to a
127+
:ref:`borrowed <c-arg-borrowed-buffer>` character string;
128+
it does not accept Unicode objects. The bytes buffer must not
113129
contain embedded null bytes; if it does, a :exc:`ValueError`
114130
exception is raised.
115131

0 commit comments

Comments
 (0)