From 28097db211202f8865966348ce51077d9df503ad Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Wed, 3 Aug 2022 15:11:05 +0100 Subject: [PATCH 1/5] Document that you shouldn't be doing your own dictionary offset calculations. --- Doc/c-api/typeobj.rst | 17 +++++------------ Doc/whatsnew/3.11.rst | 4 ++++ 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 7514801f2d4d59..57e58b2ff8a89f 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1709,18 +1709,11 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is at the very end of the structure. - The real dictionary offset in an instance can be computed from a negative - :c:member:`~PyTypeObject.tp_dictoffset` as follows:: - - dictoffset = tp_basicsize + abs(ob_size)*tp_itemsize + tp_dictoffset - if dictoffset is not aligned on sizeof(void*): - round up to sizeof(void*) - - where :c:member:`~PyTypeObject.tp_basicsize`, :c:member:`~PyTypeObject.tp_itemsize` and :c:member:`~PyTypeObject.tp_dictoffset` are - taken from the type object, and :attr:`ob_size` is taken from the instance. The - absolute value is taken because ints use the sign of :attr:`ob_size` to - store the sign of the number. (There's never a need to do this calculation - yourself; it is done for you by :c:func:`_PyObject_GetDictPtr`.) + The :c:member:`~PyTypeObject.tp_dictoffset` should be regarded as write-only. + To get the pointer to the dictionary call :c:func:`_PyObject_GetDictPtr`. + Calling :c:func:`_PyObject_GetDictPtr` may need to allocate memory for the + dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr` + when accessing an attribute on the object. **Inheritance:** diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index 1b3a685dbacb91..ecb3fe65945cde 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -1498,6 +1498,10 @@ Changes in the Python API :func:`compile` and other related functions. If invalid positions are detected, a :exc:`ValueError` will be raised. (Contributed by Pablo Galindo in :gh:`93351`) +* :c:member:`~PyTypeObject.tp_dictoffset` should be treated as write-only. + It can be set to describe C extension clases to the VM, but should be regarded + as meaningless when read. To get the pointer to the object's dictionary call + :c:func:`_PyObject_GetDictPtr` instead. Build Changes ============= From c395f45a2147c8e41ccf01e2739936cc24bb587e Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 4 Aug 2022 18:57:06 +0200 Subject: [PATCH 2/5] Document _PyObject_GetDictPtr (#13) --- Doc/c-api/object.rst | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 07a625bac02fc4..0d3e3e7ee1daac 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -137,6 +137,16 @@ Object Protocol .. versionadded:: 3.3 +.. c:function:: PyObject** _PyObject_GetDictPtr(PyObject *obj) + + Return a pointer to :py:attr:`~object.__dict__` of the object *obj*. + If there is no ``__dict__``, return ``NULL`` without setting an exception. + + Calling this function` may need to allocate memory for the + dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr` + when accessing an attribute on the object. + + .. c:function:: PyObject* PyObject_RichCompare(PyObject *o1, PyObject *o2, int opid) Compare the values of *o1* and *o2* using the operation specified by *opid*, From c20430cecbed877e16806a8ffdda11085248c863 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 4 Aug 2022 19:00:05 +0200 Subject: [PATCH 3/5] Prefer using PyObject_GenericGetDict to get the __dict__ (#14) --- Doc/c-api/object.rst | 8 ++++++++ Doc/c-api/typeobj.rst | 4 ++-- Objects/object.c | 6 +++++- 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 0d3e3e7ee1daac..58eccec0999517 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -126,6 +126,14 @@ Object Protocol A generic implementation for the getter of a ``__dict__`` descriptor. It creates the dictionary if necessary. + This function may also be called to get the :py:attr:`~object.__dict__` + of the object *o*. Pass ``NULL`` for *context* when calling it. + Since this function may need to allocate memory for theg + dictionary, it is may be more efficient to call :c:func:`PyObject_GetAttr` + when accessing an attribute on the object. + + On failure, returns ``NULL`` with an exception set. + .. versionadded:: 3.3 diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 57e58b2ff8a89f..67e8a119578fff 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -1710,8 +1710,8 @@ and :c:type:`PyType_Type` effectively act as defaults.) at the very end of the structure. The :c:member:`~PyTypeObject.tp_dictoffset` should be regarded as write-only. - To get the pointer to the dictionary call :c:func:`_PyObject_GetDictPtr`. - Calling :c:func:`_PyObject_GetDictPtr` may need to allocate memory for the + To get the pointer to the dictionary call :c:func:`PyObject_GenericGetDict`. + Calling :c:func:`PyObject_GenericGetDict` may need to allocate memory for the dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object. diff --git a/Objects/object.c b/Objects/object.c index f0c0434fab3d1e..a90c6faf99db48 100644 --- a/Objects/object.c +++ b/Objects/object.c @@ -1079,7 +1079,11 @@ _PyObject_ComputedDictPointer(PyObject *obj) /* Helper to get a pointer to an object's __dict__ slot, if any. * Creates the dict from inline attributes if necessary. - * Does not set an exception. */ + * Does not set an exception. + * + * Note that the tp_dictoffset docs used to recommend this function, + * so it should be treated as part of the public API. + */ PyObject ** _PyObject_GetDictPtr(PyObject *obj) { From 17e788d5a545f55980fffa716b15ad2e5ab1721c Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Fri, 5 Aug 2022 11:22:29 +0100 Subject: [PATCH 4/5] Update Doc/whatsnew/3.11.rst Co-authored-by: Petr Viktorin --- Doc/whatsnew/3.11.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst index ecb3fe65945cde..c9587563654745 100644 --- a/Doc/whatsnew/3.11.rst +++ b/Doc/whatsnew/3.11.rst @@ -1501,7 +1501,7 @@ Changes in the Python API * :c:member:`~PyTypeObject.tp_dictoffset` should be treated as write-only. It can be set to describe C extension clases to the VM, but should be regarded as meaningless when read. To get the pointer to the object's dictionary call - :c:func:`_PyObject_GetDictPtr` instead. + :c:func:`PyObject_GenericGetDict` instead. Build Changes ============= From 71f84b4b95d9da77e638953680915a18af3f73d6 Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Mon, 8 Aug 2022 18:38:18 +0100 Subject: [PATCH 5/5] Apply suggestions from code review Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com> --- Doc/c-api/object.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index 58eccec0999517..fb03366056b0d2 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -128,8 +128,8 @@ Object Protocol This function may also be called to get the :py:attr:`~object.__dict__` of the object *o*. Pass ``NULL`` for *context* when calling it. - Since this function may need to allocate memory for theg - dictionary, it is may be more efficient to call :c:func:`PyObject_GetAttr` + Since this function may need to allocate memory for the + dictionary, it may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object. On failure, returns ``NULL`` with an exception set. @@ -150,8 +150,8 @@ Object Protocol Return a pointer to :py:attr:`~object.__dict__` of the object *obj*. If there is no ``__dict__``, return ``NULL`` without setting an exception. - Calling this function` may need to allocate memory for the - dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr` + This function may need to allocate memory for the + dictionary, so it may be more efficient to call :c:func:`PyObject_GetAttr` when accessing an attribute on the object.