From fde9cf7266d13cfd19ff419859c3609181d6a32e Mon Sep 17 00:00:00 2001 From: Jeroen Demeyer Date: Tue, 4 Jun 2019 17:58:30 +0200 Subject: [PATCH 1/5] bpo-36974: separate docs for call protocol --- Doc/c-api/abstract.rst | 1 + Doc/c-api/call.rst | 246 +++++++++++++++++++++++++++++++++++++++++ Doc/c-api/object.rst | 240 ---------------------------------------- 3 files changed, 247 insertions(+), 240 deletions(-) create mode 100644 Doc/c-api/call.rst diff --git a/Doc/c-api/abstract.rst b/Doc/c-api/abstract.rst index 0edd1d5f6240af..1823f9d70c79f3 100644 --- a/Doc/c-api/abstract.rst +++ b/Doc/c-api/abstract.rst @@ -18,6 +18,7 @@ but whose items have not been set to some non-\ ``NULL`` value yet. .. toctree:: object.rst + call.rst number.rst sequence.rst mapping.rst diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst new file mode 100644 index 00000000000000..0921215a850371 --- /dev/null +++ b/Doc/c-api/call.rst @@ -0,0 +1,246 @@ +.. highlight:: c + +.. _call: + +Call Protocol +============= + +.. c:function:: int PyCallable_Check(PyObject *o) + + Determine if the object *o* is callable. Return ``1`` if the object is callable + and ``0`` otherwise. This function always succeeds. + + +.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable) + + Call a callable Python object *callable* without any arguments. It is the + most efficient way to call a callable Python object without any argument. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg) + + Call a callable Python object *callable* with exactly 1 positional argument + *arg* and no keyword arguments. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs) + + Call a callable Python object *callable*, with arguments given by the + tuple *args*, and named arguments given by the dictionary *kwargs*. + + *args* must not be *NULL*, use an empty tuple if no arguments are needed. + If no named arguments are needed, *kwargs* can be *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: + ``callable(*args, **kwargs)``. + + +.. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args) + + Call a callable Python object *callable*, with arguments given by the + tuple *args*. If no arguments are needed, then *args* can be *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: ``callable(*args)``. + + +.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...) + + Call a callable Python object *callable*, with a variable number of C arguments. + The C arguments are described using a :c:func:`Py_BuildValue` style format + string. The format can be *NULL*, indicating that no arguments are provided. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: ``callable(*args)``. + + Note that if you only pass :c:type:`PyObject \*` args, + :c:func:`PyObject_CallFunctionObjArgs` is a faster alternative. + + .. versionchanged:: 3.4 + The type of *format* was changed from ``char *``. + + +.. c:function:: PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...) + + Call the method named *name* of object *obj* with a variable number of C + arguments. The C arguments are described by a :c:func:`Py_BuildValue` format + string that should produce a tuple. + + The format can be *NULL*, indicating that no arguments are provided. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: + ``obj.name(arg1, arg2, ...)``. + + Note that if you only pass :c:type:`PyObject \*` args, + :c:func:`PyObject_CallMethodObjArgs` is a faster alternative. + + .. versionchanged:: 3.4 + The types of *name* and *format* were changed from ``char *``. + + +.. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL) + + Call a callable Python object *callable*, with a variable number of + :c:type:`PyObject\*` arguments. The arguments are provided as a variable number + of parameters followed by *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This is the equivalent of the Python expression: + ``callable(arg1, arg2, ...)``. + + +.. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ..., NULL) + + Calls a method of the Python object *obj*, where the name of the method is given as a + Python string object in *name*. It is called with a variable number of + :c:type:`PyObject\*` arguments. The arguments are provided as a variable number + of parameters followed by *NULL*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + +.. c:function:: PyObject* _PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name) + + Call a method of the Python object *obj* without arguments, + where the name of the method is given as a Python string object in *name*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* _PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg) + + Call a method of the Python object *obj* with a single positional argument + *arg*, where the name of the method is given as a Python string object in + *name*. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) + + Call a callable Python object *callable*, using + :c:data:`vectorcall ` if possible. + + *args* is a C array with the positional arguments. + + *nargsf* is the number of positional arguments plus optionally the flag + :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` (see below). + To get actual number of arguments, use + :c:func:`PyVectorcall_NARGS(nargsf) `. + + *kwnames* can be either NULL (no keyword arguments) or a tuple of keyword + names, which must be strings. In the latter case, the values of the keyword + arguments are stored in *args* after the positional arguments. + The number of keyword arguments does not influence *nargsf*. + + *kwnames* must contain only objects of type ``str`` (not a subclass), + and all keys must be unique. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + This uses the vectorcall protocol if the callable supports it; + otherwise, the arguments are converted to use + :c:member:`~PyTypeObject.tp_call`. + + .. note:: + + This function is provisional and expected to become public in Python 3.9, + with a different name and, possibly, changed semantics. + If you use the function, plan for updating your code for Python 3.9. + + .. versionadded:: 3.8 + +.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET + + If set in a vectorcall *nargsf* argument, the callee is allowed to + temporarily change ``args[-1]``. In other words, *args* points to + argument 1 (not 0) in the allocated vector. + The callee must restore the value of ``args[-1]`` before returning. + + For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that + ``args[0]`` may be changed. + + Whenever they can do so cheaply (without additional allocation), callers + are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. + Doing so will allow callables such as bound methods to make their onward + calls (which include a prepended *self* argument) cheaply. + + .. versionadded:: 3.8 + +.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf) + + Given a vectorcall *nargsf* argument, return the actual number of + arguments. + Currently equivalent to ``nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET``. + + .. versionadded:: 3.8 + +.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) + + Same as :c:func:`_PyObject_Vectorcall` except that the keyword arguments + are passed as a dictionary in *kwdict*. This may be *NULL* if there + are no keyword arguments. + + For callables supporting :c:data:`vectorcall `, + the arguments are internally converted to the vectorcall convention. + Therefore, this function adds some overhead compared to + :c:func:`_PyObject_Vectorcall`. + It should only be used if the caller already has a dictionary ready to use. + + .. note:: + + This function is provisional and expected to become public in Python 3.9, + with a different name and, possibly, changed semantics. + If you use the function, plan for updating your code for Python 3.9. + + .. versionadded:: 3.8 + +.. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames) + + Call a method using the vectorcall calling convention. The name of the method + is given as Python string *name*. The object whose method is called is + *args[0]* and the *args* array starting at *args[1]* represents the arguments + of the call. There must be at least one positional argument. + *nargsf* is the number of positional arguments including *args[0]*, + plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may + temporarily be changed. Keyword arguments can be passed just like in + :c:func:`_PyObject_Vectorcall`. + + If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, + this will actually call the unbound method object with the full + *args* vector as arguments. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst index fd1e9c65aaba97..dda2d5e2ad3105 100644 --- a/Doc/c-api/object.rst +++ b/Doc/c-api/object.rst @@ -247,246 +247,6 @@ Object Protocol of base classes). -.. c:function:: int PyCallable_Check(PyObject *o) - - Determine if the object *o* is callable. Return ``1`` if the object is callable - and ``0`` otherwise. This function always succeeds. - - -.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable) - - Call a callable Python object *callable* without any arguments. It is the - most efficient way to call a callable Python object without any argument. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg) - - Call a callable Python object *callable* with exactly 1 positional argument - *arg* and no keyword arguments. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs) - - Call a callable Python object *callable*, with arguments given by the - tuple *args*, and named arguments given by the dictionary *kwargs*. - - *args* must not be *NULL*, use an empty tuple if no arguments are needed. - If no named arguments are needed, *kwargs* can be *NULL*. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - This is the equivalent of the Python expression: - ``callable(*args, **kwargs)``. - - -.. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args) - - Call a callable Python object *callable*, with arguments given by the - tuple *args*. If no arguments are needed, then *args* can be *NULL*. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - This is the equivalent of the Python expression: ``callable(*args)``. - - -.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...) - - Call a callable Python object *callable*, with a variable number of C arguments. - The C arguments are described using a :c:func:`Py_BuildValue` style format - string. The format can be *NULL*, indicating that no arguments are provided. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - This is the equivalent of the Python expression: ``callable(*args)``. - - Note that if you only pass :c:type:`PyObject \*` args, - :c:func:`PyObject_CallFunctionObjArgs` is a faster alternative. - - .. versionchanged:: 3.4 - The type of *format* was changed from ``char *``. - - -.. c:function:: PyObject* PyObject_CallMethod(PyObject *obj, const char *name, const char *format, ...) - - Call the method named *name* of object *obj* with a variable number of C - arguments. The C arguments are described by a :c:func:`Py_BuildValue` format - string that should produce a tuple. - - The format can be *NULL*, indicating that no arguments are provided. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - This is the equivalent of the Python expression: - ``obj.name(arg1, arg2, ...)``. - - Note that if you only pass :c:type:`PyObject \*` args, - :c:func:`PyObject_CallMethodObjArgs` is a faster alternative. - - .. versionchanged:: 3.4 - The types of *name* and *format* were changed from ``char *``. - - -.. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL) - - Call a callable Python object *callable*, with a variable number of - :c:type:`PyObject\*` arguments. The arguments are provided as a variable number - of parameters followed by *NULL*. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - This is the equivalent of the Python expression: - ``callable(arg1, arg2, ...)``. - - -.. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ..., NULL) - - Calls a method of the Python object *obj*, where the name of the method is given as a - Python string object in *name*. It is called with a variable number of - :c:type:`PyObject\*` arguments. The arguments are provided as a variable number - of parameters followed by *NULL*. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - -.. c:function:: PyObject* _PyObject_CallMethodNoArgs(PyObject *obj, PyObject *name) - - Call a method of the Python object *obj* without arguments, - where the name of the method is given as a Python string object in *name*. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* _PyObject_CallMethodOneArg(PyObject *obj, PyObject *name, PyObject *arg) - - Call a method of the Python object *obj* with a single positional argument - *arg*, where the name of the method is given as a Python string object in - *name*. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - .. versionadded:: 3.9 - - -.. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - - Call a callable Python object *callable*, using - :c:data:`vectorcall ` if possible. - - *args* is a C array with the positional arguments. - - *nargsf* is the number of positional arguments plus optionally the flag - :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` (see below). - To get actual number of arguments, use - :c:func:`PyVectorcall_NARGS(nargsf) `. - - *kwnames* can be either NULL (no keyword arguments) or a tuple of keyword - names, which must be strings. In the latter case, the values of the keyword - arguments are stored in *args* after the positional arguments. - The number of keyword arguments does not influence *nargsf*. - - *kwnames* must contain only objects of type ``str`` (not a subclass), - and all keys must be unique. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - This uses the vectorcall protocol if the callable supports it; - otherwise, the arguments are converted to use - :c:member:`~PyTypeObject.tp_call`. - - .. note:: - - This function is provisional and expected to become public in Python 3.9, - with a different name and, possibly, changed semantics. - If you use the function, plan for updating your code for Python 3.9. - - .. versionadded:: 3.8 - -.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET - - If set in a vectorcall *nargsf* argument, the callee is allowed to - temporarily change ``args[-1]``. In other words, *args* points to - argument 1 (not 0) in the allocated vector. - The callee must restore the value of ``args[-1]`` before returning. - - For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that - ``args[0]`` may be changed. - - Whenever they can do so cheaply (without additional allocation), callers - are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. - Doing so will allow callables such as bound methods to make their onward - calls (which include a prepended *self* argument) cheaply. - - .. versionadded:: 3.8 - -.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf) - - Given a vectorcall *nargsf* argument, return the actual number of - arguments. - Currently equivalent to ``nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET``. - - .. versionadded:: 3.8 - -.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) - - Same as :c:func:`_PyObject_Vectorcall` except that the keyword arguments - are passed as a dictionary in *kwdict*. This may be *NULL* if there - are no keyword arguments. - - For callables supporting :c:data:`vectorcall `, - the arguments are internally converted to the vectorcall convention. - Therefore, this function adds some overhead compared to - :c:func:`_PyObject_Vectorcall`. - It should only be used if the caller already has a dictionary ready to use. - - .. note:: - - This function is provisional and expected to become public in Python 3.9, - with a different name and, possibly, changed semantics. - If you use the function, plan for updating your code for Python 3.9. - - .. versionadded:: 3.8 - -.. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames) - - Call a method using the vectorcall calling convention. The name of the method - is given as Python string *name*. The object whose method is called is - *args[0]* and the *args* array starting at *args[1]* represents the arguments - of the call. There must be at least one positional argument. - *nargsf* is the number of positional arguments including *args[0]*, - plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may - temporarily be changed. Keyword arguments can be passed just like in - :c:func:`_PyObject_Vectorcall`. - - If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, - this will actually call the unbound method object with the full - *args* vector as arguments. - - Return the result of the call on success, or raise an exception and return - *NULL* on failure. - - .. versionadded:: 3.9 - .. c:function:: Py_hash_t PyObject_Hash(PyObject *o) .. index:: builtin: hash From 784fb453c18964d31c614c3780e122cddef0d4f9 Mon Sep 17 00:00:00 2001 From: Jeroen Demeyer Date: Tue, 4 Jun 2019 18:15:08 +0200 Subject: [PATCH 2/5] bpo-36974: expand call protocol documentation --- Doc/c-api/call.rst | 190 +++++++++++++++++++++++++++++---------- Doc/c-api/exceptions.rst | 4 + Doc/c-api/structures.rst | 3 +- Doc/c-api/type.rst | 3 +- Doc/c-api/typeobj.rst | 88 +++++++++--------- Doc/whatsnew/3.8.rst | 17 ++-- 6 files changed, 206 insertions(+), 99 deletions(-) diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index 0921215a850371..e2748fd96f9bec 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -5,6 +5,109 @@ Call Protocol ============= +CPython supports two different calling protocols: +*tp_call* and vectorcall. + +The *tp_call* Protocol +---------------------- + +Classes can implement callables by setting :c:member:`~PyTypeObject.tp_call`. + +A call is made using a tuple for the positional arguments +and a dict for the keyword arguments, just like ``f(*args, **kwargs)`` +in the Python language. +*args* must be non-NULL +(use an empty tuple if there are no arguments) +but *kwargs* may be *NULL* if there are no keyword arguments. + +This convention is not only used by *tp_call*: +also :c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init` +pass arguments this way +(apart from the special handling of *subtype* and *self*). + +For making a call this way, use :c:func:`PyObject_Call`. + +.. _vectorcall: + +The Vectorcall Protocol +----------------------- + +.. versionadded:: 3.8 + +The vectorcall protocol was introduced in :pep:`590` +for making calls more efficient. + +Classes can implement the vectorcall protocol by enabling the +:const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag and setting +:c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the +object structure where a *vectorcallfunc* appears. +This is a function with the following signature: + +.. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) + + - *callable* is the object being called. + - *args* is a C array consisting of the positional arguments followed by the + values of the keyword arguments. + This can be *NULL* if there are no arguments. + - *nargsf* is the number of positional arguments plus possibly the + :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. + To get the actual number of positional arguments from *nargsf*, + use :c:func:`PyVectorcall_NARGS`. + - *kwnames* is a tuple containing the names of the keyword arguments, + in other words the keys of the kwargs dict. + These names must be strings (instances of ``str`` or a subclass) + and they must be unique. + If there are no keyword arguments, then *kwnames* can instead be *NULL*. + +.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET + + If this flag is set in a vectorcall *nargsf* argument, the callee is allowed + to temporarily change ``args[-1]``. In other words, *args* points to + argument 1 (not 0) in the allocated vector. + The callee must restore the value of ``args[-1]`` before returning. + + For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that + ``args[0]`` may be changed. + + Whenever they can do so cheaply (without additional allocation), callers + are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. + Doing so will allow callables such as bound methods to make their onward + calls (which include a prepended *self* argument) very efficiently. + +.. warning:: + + As rule of thumb, CPython will internally use the vectorcall + protocol if the callable supports it. However, this is not a hard rule, + *tp_call* may be used instead. + Therefore, a class supporting vectorcall must also set + :c:member:`~PyTypeObject.tp_call`. + Moreover, the callable must behave the same + regardless of which protocol is used. + The recommended way to achieve this is by setting + :c:member:`~PyTypeObject.tp_call` to :c:func:`PyVectorcall_Call`. + + An object should not implement vectorcall if that would be slower + than *tp_call*. For example, if the callee needs to convert + the arguments to an args tuple and kwargs dict anyway, then there is no point + in implementing vectorcall. + +For making a vectorcall, use :c:func:`_PyObject_Vectorcall`. + +Recursion Control +----------------- + +When using *tp_call*, callees do not need to worry about +:ref:`recursion `: CPython uses +:c:func:`Py_EnterRecursiveCall` and :c:func:`Py_LeaveRecursiveCall` +for calls made using *tp_call*. + +For efficiency, this is not the case for calls done using vectorcall: +the callee should use *Py_EnterRecursiveCall* and *Py_LeaveRecursiveCall* +if needed. + +C API Functions +--------------- + .. c:function:: int PyCallable_Check(PyObject *o) Determine if the object *o* is callable. Return ``1`` if the object is callable @@ -147,31 +250,14 @@ Call Protocol .. c:function:: PyObject* _PyObject_Vectorcall(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - Call a callable Python object *callable*, using - :c:data:`vectorcall ` if possible. - - *args* is a C array with the positional arguments. - - *nargsf* is the number of positional arguments plus optionally the flag - :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` (see below). - To get actual number of arguments, use - :c:func:`PyVectorcall_NARGS(nargsf) `. - - *kwnames* can be either NULL (no keyword arguments) or a tuple of keyword - names, which must be strings. In the latter case, the values of the keyword - arguments are stored in *args* after the positional arguments. - The number of keyword arguments does not influence *nargsf*. - - *kwnames* must contain only objects of type ``str`` (not a subclass), - and all keys must be unique. + Call a callable Python object *callable*. + The arguments are the same as for :c:type:`vectorcallfunc`. + If *callable* supports vectorcall_, this directly calls + the vectorcall function stored in *callable*. Return the result of the call on success, or raise an exception and return *NULL* on failure. - This uses the vectorcall protocol if the callable supports it; - otherwise, the arguments are converted to use - :c:member:`~PyTypeObject.tp_call`. - .. note:: This function is provisional and expected to become public in Python 3.9, @@ -180,20 +266,23 @@ Call Protocol .. versionadded:: 3.8 -.. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET +.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) - If set in a vectorcall *nargsf* argument, the callee is allowed to - temporarily change ``args[-1]``. In other words, *args* points to - argument 1 (not 0) in the allocated vector. - The callee must restore the value of ``args[-1]`` before returning. + Call *callable* with positional arguments passed exactly as in the vectorcall_ protocol + but with keyword arguments passed as a dictionary *kwdict*. + The *args* array contains only the positional arguments. - For :c:func:`_PyObject_VectorcallMethod`, this flag means instead that - ``args[0]`` may be changed. + Regardless of which protocol is used internally, + a conversion of arguments needs to be done. + Therefore, this function should only be used if the caller + already has a dictionary ready to use for the keyword arguments, + but not a tuple for the positional arguments. - Whenever they can do so cheaply (without additional allocation), callers - are encouraged to use :const:`PY_VECTORCALL_ARGUMENTS_OFFSET`. - Doing so will allow callables such as bound methods to make their onward - calls (which include a prepended *self* argument) cheaply. + .. note:: + + This function is provisional and expected to become public in Python 3.9, + with a different name and, possibly, changed semantics. + If you use the function, plan for updating your code for Python 3.9. .. versionadded:: 3.8 @@ -201,27 +290,36 @@ Call Protocol Given a vectorcall *nargsf* argument, return the actual number of arguments. - Currently equivalent to ``nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET``. + Currently equivalent to:: + + (Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET) + + However, the function ``PyVectorcall_NARGS`` should be used to allow + for future extensions. .. versionadded:: 3.8 -.. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) +.. c:function:: vectorcallfunc _PyVectorcall_Function(PyObject *op) - Same as :c:func:`_PyObject_Vectorcall` except that the keyword arguments - are passed as a dictionary in *kwdict*. This may be *NULL* if there - are no keyword arguments. + If *op* does not support the vectorcall protocol (either because the type + does not or because the specific instance does not), return *NULL*. + Otherwise, return the vectorcall function pointer stored in *op*. + This function never sets an exception. - For callables supporting :c:data:`vectorcall `, - the arguments are internally converted to the vectorcall convention. - Therefore, this function adds some overhead compared to - :c:func:`_PyObject_Vectorcall`. - It should only be used if the caller already has a dictionary ready to use. + This is mostly useful to check whether or not *op* supports vectorcall, + which can be done by checking ``_PyVectorcall_Function(op) != NULL``. - .. note:: + .. versionadded:: 3.8 - This function is provisional and expected to become public in Python 3.9, - with a different name and, possibly, changed semantics. - If you use the function, plan for updating your code for Python 3.9. +.. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) + + Call *callable*'s :c:type:`vectorcallfunc` with positional and keyword + arguments given in a tuple and dict, respectively. + + This is a specialized function, intended to be put in the + :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. + It does not check the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag + and it does not fall back to ``tp_call``. .. versionadded:: 3.8 diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index d8173935596bb8..2fa4cd9420c54f 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -695,6 +695,8 @@ The following functions are used to create and modify Unicode exceptions from C. ``0`` on success, ``-1`` on failure. +.. _recursion: + Recursion Control ================= @@ -702,6 +704,8 @@ These two functions provide a way to perform safe recursive calls at the C level, both in the core and in extension modules. They are needed if the recursive code does not necessarily invoke Python code (which tracks its recursion depth automatically). +It's also not needed for *tp_call* implementations +because the :ref:`call protocol ` takes care of recursion handling. .. c:function:: int Py_EnterRecursiveCall(const char *where) diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst index d4e65afef14d53..200ee8a5a18e89 100644 --- a/Doc/c-api/structures.rst +++ b/Doc/c-api/structures.rst @@ -201,7 +201,8 @@ also keyword arguments. So there are a total of 6 calling conventions: Extension of :const:`METH_FASTCALL` supporting also keyword arguments, with methods of type :c:type:`_PyCFunctionFastWithKeywords`. - Keyword arguments are passed the same way as in the vectorcall protocol: + Keyword arguments are passed the same way as in the + :ref:`vectorcall protocol `: there is an additional fourth :c:type:`PyObject\*` parameter which is a tuple representing the names of the keyword arguments (which are guaranteed to be strings) diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst index 8f8367ab77c8c4..2e3c4f37e6401d 100644 --- a/Doc/c-api/type.rst +++ b/Doc/c-api/type.rst @@ -191,9 +191,10 @@ The following functions and structs are used to create * :c:member:`~PyTypeObject.tp_cache` * :c:member:`~PyTypeObject.tp_subclasses` * :c:member:`~PyTypeObject.tp_weaklist` - * :c:member:`~PyTypeObject.tp_print` * :c:member:`~PyTypeObject.tp_weaklistoffset` * :c:member:`~PyTypeObject.tp_dictoffset` + * :c:member:`~PyTypeObject.tp_vectorcall_offset` + * :c:member:`~PyTypeObject.tp_vectorcall` * :c:member:`~PyBufferProcs.bf_getbuffer` * :c:member:`~PyBufferProcs.bf_releasebuffer` diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst index 638fb0c32d788b..3a2b6560c59e2f 100644 --- a/Doc/c-api/typeobj.rst +++ b/Doc/c-api/typeobj.rst @@ -49,7 +49,7 @@ Quick Reference +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_dealloc` | :c:type:`destructor` | | X | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ - | :c:member:`~PyTypeObject.tp_vectorcall_offset` | Py_ssize_t | | | | | ? | + | :c:member:`~PyTypeObject.tp_vectorcall_offset` | Py_ssize_t | | | X | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | (:c:member:`~PyTypeObject.tp_getattr`) | :c:type:`getattrfunc` | __getattribute__, | | | | G | | | | __getattr__ | | | | | @@ -145,6 +145,8 @@ Quick Reference +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ | :c:member:`~PyTypeObject.tp_finalize` | :c:type:`destructor` | __del__ | | | | X | +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ + | :c:member:`~PyTypeObject.tp_vectorcall` | :c:type:`vectorcallfunc` | | | | | | + +------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+ If :const:`COUNT_ALLOCS` is defined then the following (internal-only) fields exist as well: @@ -672,42 +674,29 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. c:member:: Py_ssize_t PyTypeObject.tp_vectorcall_offset An optional offset to a per-instance function that implements calling - the object using the *vectorcall* protocol, a more efficient alternative + the object using the :ref:`vectorcall protocol `, + a more efficient alternative of the simpler :c:member:`~PyTypeObject.tp_call`. This field is only used if the flag :const:`_Py_TPFLAGS_HAVE_VECTORCALL` is set. If so, this must be a positive integer containing the offset in the instance of a :c:type:`vectorcallfunc` pointer. - The signature is the same as for :c:func:`_PyObject_Vectorcall`:: - - PyObject *vectorcallfunc(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - The *vectorcallfunc* pointer may be zero, in which case the instance behaves + The *vectorcallfunc* pointer may be *NULL*, in which case the instance behaves as if :const:`_Py_TPFLAGS_HAVE_VECTORCALL` was not set: calling the instance falls back to :c:member:`~PyTypeObject.tp_call`. Any class that sets ``_Py_TPFLAGS_HAVE_VECTORCALL`` must also set :c:member:`~PyTypeObject.tp_call` and make sure its behaviour is consistent with the *vectorcallfunc* function. - This can be done by setting *tp_call* to ``PyVectorcall_Call``: - - .. c:function:: PyObject *PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) + This can be done by setting *tp_call* to :c:func:`PyVectorcall_Call`. - Call *callable*'s *vectorcallfunc* with positional and keyword - arguments given in a tuple and dict, respectively. - - This function is intended to be used in the ``tp_call`` slot. - It does not fall back to ``tp_call`` and it currently does not check the - ``_Py_TPFLAGS_HAVE_VECTORCALL`` flag. - To call an object, use one of the :c:func:`PyObject_Call ` - functions instead. - - .. note:: + .. warning:: It is not recommended for :ref:`heap types ` to implement the vectorcall protocol. - When a user sets ``__call__`` in Python code, only ``tp_call`` is updated, - possibly making it inconsistent with the vectorcall function. + When a user sets :attr:`__call__` in Python code, only *tp_call* is updated, + likely making it inconsistent with the vectorcall function. .. note:: @@ -717,18 +706,19 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. versionchanged:: 3.8 - This slot was used for print formatting in Python 2.x. - In Python 3.0 to 3.7, it was reserved and named ``tp_print``. + Before version 3.8, this slot was named ``tp_print``. + In Python 2.x, it was used for printing to a file. + In Python 3.0 to 3.7, it was unused. **Inheritance:** - This field is inherited by subtypes together with - :c:member:`~PyTypeObject.tp_call`: a subtype inherits - :c:member:`~PyTypeObject.tp_vectorcall_offset` from its base type when - the subtype’s :c:member:`~PyTypeObject.tp_call` is NULL. - - Note that `heap types`_ (including subclasses defined in Python) do not - inherit the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag. + This field is always inherited. + However, the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag is not + always inherited. If it's not, then the subclass won't use + :ref:`vectorcall `, except when + :c:func:`PyVectorcall_Call` is explicitly called. + This is in particular the case for `heap types`_ + (including subclasses defined in Python). .. c:member:: getattrfunc PyTypeObject.tp_getattr @@ -1155,18 +1145,17 @@ and :c:type:`PyType_Type` effectively act as defaults.) :c:member:`~PyTypeObject.tp_finalize` slot is always present in the type structure. + .. data:: _Py_TPFLAGS_HAVE_VECTORCALL - This bit is set when the class implements the vectorcall protocol. + This bit is set when the class implements + the :ref:`vectorcall protocol `. See :c:member:`~PyTypeObject.tp_vectorcall_offset` for details. **Inheritance:** - This bit is set on *static* subtypes if ``tp_flags`` is not overridden: - a subtype inherits ``_Py_TPFLAGS_HAVE_VECTORCALL`` from its base type - when the subtype’s :c:member:`~PyTypeObject.tp_call` is NULL - and the subtype's ``Py_TPFLAGS_HEAPTYPE`` is not set. - + This bit is inherited for *static* subtypes if + :c:member:`~PyTypeObject.tp_call` is also inherited. `Heap types`_ do not inherit ``_Py_TPFLAGS_HAVE_VECTORCALL``. .. note:: @@ -1705,9 +1694,9 @@ and :c:type:`PyType_Type` effectively act as defaults.) PyObject *tp_new(PyTypeObject *subtype, PyObject *args, PyObject *kwds); - The subtype argument is the type of the object being created; the *args* and + The *subtype* argument is the type of the object being created; the *args* and *kwds* arguments represent positional and keyword arguments of the call to the - type. Note that subtype doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new` + type. Note that *subtype* doesn't have to equal the type whose :c:member:`~PyTypeObject.tp_new` function is called; it may be a subtype of that type (but not an unrelated type). @@ -1890,6 +1879,21 @@ and :c:type:`PyType_Type` effectively act as defaults.) .. seealso:: "Safe object finalization" (:pep:`442`) +.. c:member:: vectorcallfunc PyTypeObject.tp_vectorcall + + Vectorcall function to use for calls of this type object. + In other words, it is used to implement + :ref:`vectorcall ` for ``type.__call__``. + If ``tp_vectorcall`` is *NULL*, the default call implementation + using :attr:`__new__` and :attr:`__init__` is used. + + **Inheritance:** + + This field is never inherited. + + .. versionadded:: 3.9 (the field exists since 3.8 but it's only used since 3.9) + + The remaining fields are only defined if the feature test macro :const:`COUNT_ALLOCS` is defined, and are for internal use only. They are documented here for completeness. None of these fields are inherited by @@ -2359,14 +2363,6 @@ Slot Type typedefs .. c:type:: void (*destructor)(PyObject *) -.. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - - See :c:member:`~PyTypeObject.tp_vectorcall_offset`. - - Arguments to ``vectorcallfunc`` are the same as for :c:func:`_PyObject_Vectorcall`. - - .. versionadded:: 3.8 - .. c:type:: void (*freefunc)(void *) See :c:member:`~PyTypeObject.tp_free`. diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index e8238251d6ea25..942a0a7d43f197 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -343,20 +343,20 @@ See :pep:`587` for a full description. (Contributed by Victor Stinner in :issue:`36763`.) -Vectorcall: a fast calling protocol for CPython ------------------------------------------------ +PEP 590: Vectorcall: a fast calling protocol for CPython +-------------------------------------------------------- -The "vectorcall" protocol is added to the Python/C API. +:ref:`vectorcall` is added to the Python/C API. It is meant to formalize existing optimizations which were already done for various classes. -Any extension type implementing a callable can use this protocol. +Any static type implementing a callable can use this protocol. This is currently provisional, the aim is to make it fully public in Python 3.9. See :pep:`590` for a full description. -(Contributed by Jeroen Demeyer and Mark Shannon in :issue:`36974`.) +(Contributed by Jeroen Demeyer, Mark Shannon and Petr Viktorin in :issue:`36974`.) Pickle protocol 5 with out-of-band data buffers @@ -1621,6 +1621,13 @@ Changes in the Python API Changes in the C API -------------------- +* In order to support the :ref:`vectorcall protocol `, + the type slot ``tp_print`` is replaced by + :c:member:`~PyTypeObject.tp_vectorcall_offset` + and a new slot :c:member:`~PyTypeObject.tp_vectorcall` is added. + Two new flags :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` + and :const:`_Py_TPFLAGS_HAVE_VECTORCALL` are added. + * The :c:type:`PyCompilerFlags` structure gets a new *cf_feature_version* field. It should be initialized to ``PY_MINOR_VERSION``. The field is ignored by default, it is used if and only if ``PyCF_ONLY_AST`` flag is set in From b93ea4b8c787d67f1a24a0144adf6db0257a30f9 Mon Sep 17 00:00:00 2001 From: Jeroen Demeyer Date: Mon, 9 Sep 2019 22:52:16 +0200 Subject: [PATCH 3/5] bpo-36974: review by Ashwin Ramaswami --- Doc/c-api/call.rst | 28 ++++++++++++++-------------- Doc/c-api/exceptions.rst | 2 +- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index e2748fd96f9bec..4d54d41b08ba1a 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -21,8 +21,8 @@ in the Python language. but *kwargs* may be *NULL* if there are no keyword arguments. This convention is not only used by *tp_call*: -also :c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init` -pass arguments this way +:c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init` +also pass arguments this way (apart from the special handling of *subtype* and *self*). For making a call this way, use :c:func:`PyObject_Call`. @@ -53,8 +53,8 @@ This is a function with the following signature: :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. To get the actual number of positional arguments from *nargsf*, use :c:func:`PyVectorcall_NARGS`. - - *kwnames* is a tuple containing the names of the keyword arguments, - in other words the keys of the kwargs dict. + - *kwnames* is a tuple containing the names of the keyword arguments; + in other words, the keys of the kwargs dict. These names must be strings (instances of ``str`` or a subclass) and they must be unique. If there are no keyword arguments, then *kwnames* can instead be *NULL*. @@ -79,7 +79,7 @@ This is a function with the following signature: As rule of thumb, CPython will internally use the vectorcall protocol if the callable supports it. However, this is not a hard rule, *tp_call* may be used instead. - Therefore, a class supporting vectorcall must also set + Therefore, a class supporting vectorcall must also implement :c:member:`~PyTypeObject.tp_call`. Moreover, the callable must behave the same regardless of which protocol is used. @@ -141,7 +141,7 @@ C API Functions Call a callable Python object *callable*, with arguments given by the tuple *args*, and named arguments given by the dictionary *kwargs*. - *args* must not be *NULL*, use an empty tuple if no arguments are needed. + *args* must not be *NULL*; use an empty tuple if no arguments are needed. If no named arguments are needed, *kwargs* can be *NULL*. Return the result of the call on success, or raise an exception and return @@ -184,7 +184,7 @@ C API Functions Call the method named *name* of object *obj* with a variable number of C arguments. The C arguments are described by a :c:func:`Py_BuildValue` format - string that should produce a tuple. + string that should produce a tuple. The format can be *NULL*, indicating that no arguments are provided. @@ -204,7 +204,7 @@ C API Functions .. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL) Call a callable Python object *callable*, with a variable number of - :c:type:`PyObject\*` arguments. The arguments are provided as a variable number + :c:type:`PyObject \*` arguments. The arguments are provided as a variable number of parameters followed by *NULL*. Return the result of the call on success, or raise an exception and return @@ -216,9 +216,9 @@ C API Functions .. c:function:: PyObject* PyObject_CallMethodObjArgs(PyObject *obj, PyObject *name, ..., NULL) - Calls a method of the Python object *obj*, where the name of the method is given as a + Call a method of the Python object *obj*, where the name of the method is given as a Python string object in *name*. It is called with a variable number of - :c:type:`PyObject\*` arguments. The arguments are provided as a variable number + :c:type:`PyObject \*` arguments. The arguments are provided as a variable number of parameters followed by *NULL*. Return the result of the call on success, or raise an exception and return @@ -268,7 +268,7 @@ C API Functions .. c:function:: PyObject* _PyObject_FastCallDict(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwdict) - Call *callable* with positional arguments passed exactly as in the vectorcall_ protocol + Call *callable* with positional arguments passed exactly as in the vectorcall_ protocol, but with keyword arguments passed as a dictionary *kwdict*. The *args* array contains only the positional arguments. @@ -304,7 +304,7 @@ C API Functions If *op* does not support the vectorcall protocol (either because the type does not or because the specific instance does not), return *NULL*. Otherwise, return the vectorcall function pointer stored in *op*. - This function never sets an exception. + This function never raises an exception. This is mostly useful to check whether or not *op* supports vectorcall, which can be done by checking ``_PyVectorcall_Function(op) != NULL``. @@ -326,8 +326,8 @@ C API Functions .. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames) Call a method using the vectorcall calling convention. The name of the method - is given as Python string *name*. The object whose method is called is - *args[0]* and the *args* array starting at *args[1]* represents the arguments + is given as a Python string *name*. The object whose method is called is + *args[0]*, and the *args* array starting at *args[1]* represents the arguments of the call. There must be at least one positional argument. *nargsf* is the number of positional arguments including *args[0]*, plus :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` if the value of ``args[0]`` may diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 2fa4cd9420c54f..3539e1d360b131 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -704,7 +704,7 @@ These two functions provide a way to perform safe recursive calls at the C level, both in the core and in extension modules. They are needed if the recursive code does not necessarily invoke Python code (which tracks its recursion depth automatically). -It's also not needed for *tp_call* implementations +They are also not needed for *tp_call* implementations because the :ref:`call protocol ` takes care of recursion handling. .. c:function:: int Py_EnterRecursiveCall(const char *where) From cadd0f887bedaa6a6e22ca14d92f82b2c1c43edd Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Tue, 10 Sep 2019 13:22:57 +0100 Subject: [PATCH 4/5] Reorganization and rewording - Repeat the tp_call signature, for clarity - Warn that vectorcall API is provisional - Make the existing warning smaller, but move the explanation up a bit - Group calling API together and provide a summary table - Put smaller vectorcall-related sections under vectorcall - Minor rewordings --- Doc/c-api/call.rst | 255 ++++++++++++++++++++++++++++----------------- 1 file changed, 159 insertions(+), 96 deletions(-) diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index 4d54d41b08ba1a..74aa2d3803bc60 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -11,21 +11,24 @@ CPython supports two different calling protocols: The *tp_call* Protocol ---------------------- -Classes can implement callables by setting :c:member:`~PyTypeObject.tp_call`. +Instances of classes that set :c:member:`~PyTypeObject.tp_call` are callable. +The signature of the slot is:: + + PyObject *tp_call(PyObject *callable, PyObject *args, PyObject *kwargs); A call is made using a tuple for the positional arguments -and a dict for the keyword arguments, just like ``f(*args, **kwargs)`` -in the Python language. -*args* must be non-NULL -(use an empty tuple if there are no arguments) +and a dict for the keyword arguments, similarly to +``callable(*args, **kwargs)`` in Python code. +*args* must be non-NULL (use an empty tuple if there are no arguments) but *kwargs* may be *NULL* if there are no keyword arguments. This convention is not only used by *tp_call*: :c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_init` -also pass arguments this way -(apart from the special handling of *subtype* and *self*). +also pass arguments this way. + +To call an object, use :c:func:`PyObject_Call` or other +:ref:`call API `. -For making a call this way, use :c:func:`PyObject_Call`. .. _vectorcall: @@ -34,30 +37,58 @@ The Vectorcall Protocol .. versionadded:: 3.8 -The vectorcall protocol was introduced in :pep:`590` +The vectorcall protocol was introduced in :pep:`590` as an additional protocol for making calls more efficient. +.. warning:: + + The vectorcall API is provisional and expected to become public in + Python 3.9, with a different name and, possibly, changed semantics. + If you use the function, plan for updating your code for Python 3.9. + +As rule of thumb, CPython will prefer the vectorcall for internal calls +if the callable supports it. However, this is not a hard rule. +Additionally, some third-party extensions use *tp_call* directly +(rather than using :c:func:`PyObject_Call`). +Therefore, a class supporting vectorcall must also implement +:c:member:`~PyTypeObject.tp_call`. +Moreover, the callable must behave the same +regardless of which protocol is used. +The recommended way to achieve this is by setting +:c:member:`~PyTypeObject.tp_call` to :c:func:`PyVectorcall_Call`. +This bears repeating: + +.. warning:: + + A class supporting vectorcall **must** also implement + :c:member:`~PyTypeObject.tp_call` with the same semantics. + +A class should not implement vectorcall if that would be slower +than *tp_call*. For example, if the callee needs to convert +the arguments to an args tuple and kwargs dict anyway, then there is no point +in implementing vectorcall. + Classes can implement the vectorcall protocol by enabling the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag and setting :c:member:`~PyTypeObject.tp_vectorcall_offset` to the offset inside the object structure where a *vectorcallfunc* appears. -This is a function with the following signature: +This is a pointer to a function with the following signature: .. c:type:: PyObject *(*vectorcallfunc)(PyObject *callable, PyObject *const *args, size_t nargsf, PyObject *kwnames) - - *callable* is the object being called. - - *args* is a C array consisting of the positional arguments followed by the - values of the keyword arguments. - This can be *NULL* if there are no arguments. - - *nargsf* is the number of positional arguments plus possibly the - :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. - To get the actual number of positional arguments from *nargsf*, - use :c:func:`PyVectorcall_NARGS`. - - *kwnames* is a tuple containing the names of the keyword arguments; - in other words, the keys of the kwargs dict. - These names must be strings (instances of ``str`` or a subclass) - and they must be unique. - If there are no keyword arguments, then *kwnames* can instead be *NULL*. +- *callable* is the object being called. +- *args* is a C array consisting of the positional arguments followed by the + values of the keyword arguments. + This can be *NULL* if there are no arguments. +- *nargsf* is the number of positional arguments plus possibly the + :const:`PY_VECTORCALL_ARGUMENTS_OFFSET` flag. + To get the actual number of positional arguments from *nargsf*, + use :c:func:`PyVectorcall_NARGS`. +- *kwnames* is a tuple containing the names of the keyword arguments; + in other words, the keys of the kwargs dict. + These names must be strings (instances of ``str`` or a subclass) + and they must be unique. + If there are no keyword arguments, then *kwnames* can instead be *NULL*. .. c:var:: PY_VECTORCALL_ARGUMENTS_OFFSET @@ -74,27 +105,13 @@ This is a function with the following signature: Doing so will allow callables such as bound methods to make their onward calls (which include a prepended *self* argument) very efficiently. -.. warning:: - - As rule of thumb, CPython will internally use the vectorcall - protocol if the callable supports it. However, this is not a hard rule, - *tp_call* may be used instead. - Therefore, a class supporting vectorcall must also implement - :c:member:`~PyTypeObject.tp_call`. - Moreover, the callable must behave the same - regardless of which protocol is used. - The recommended way to achieve this is by setting - :c:member:`~PyTypeObject.tp_call` to :c:func:`PyVectorcall_Call`. - - An object should not implement vectorcall if that would be slower - than *tp_call*. For example, if the callee needs to convert - the arguments to an args tuple and kwargs dict anyway, then there is no point - in implementing vectorcall. +To call an object that implements vectorcall, use a :ref:`call API ` +function as with any other callable. +:c:func:`_PyObject_Vectorcall` will usually be most efficient. -For making a vectorcall, use :c:func:`_PyObject_Vectorcall`. Recursion Control ------------------ +................. When using *tp_call*, callees do not need to worry about :ref:`recursion `: CPython uses @@ -105,35 +122,87 @@ For efficiency, this is not the case for calls done using vectorcall: the callee should use *Py_EnterRecursiveCall* and *Py_LeaveRecursiveCall* if needed. -C API Functions ---------------- -.. c:function:: int PyCallable_Check(PyObject *o) +Vectorcall Support API +...................... - Determine if the object *o* is callable. Return ``1`` if the object is callable - and ``0`` otherwise. This function always succeeds. +.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf) + Given a vectorcall *nargsf* argument, return the actual number of + arguments. + Currently equivalent to:: -.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable) + (Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET) - Call a callable Python object *callable* without any arguments. It is the - most efficient way to call a callable Python object without any argument. + However, the function ``PyVectorcall_NARGS`` should be used to allow + for future extensions. - Return the result of the call on success, or raise an exception and return - *NULL* on failure. + .. versionadded:: 3.8 - .. versionadded:: 3.9 +.. c:function:: vectorcallfunc _PyVectorcall_Function(PyObject *op) + If *op* does not support the vectorcall protocol (either because the type + does not or because the specific instance does not), return *NULL*. + Otherwise, return the vectorcall function pointer stored in *op*. + This function never raises an exception. -.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg) + This is mostly useful to check whether or not *op* supports vectorcall, + which can be done by checking ``_PyVectorcall_Function(op) != NULL``. - Call a callable Python object *callable* with exactly 1 positional argument - *arg* and no keyword arguments. + .. versionadded:: 3.8 - Return the result of the call on success, or raise an exception and return - *NULL* on failure. +.. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) - .. versionadded:: 3.9 + Call *callable*'s :c:type:`vectorcallfunc` with positional and keyword + arguments given in a tuple and dict, respectively. + + This is a specialized function, intended to be put in the + :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. + It does not check the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag + and it does not fall back to ``tp_call``. + + .. versionadded:: 3.8 + + +.. _capi-call: + +Object Calling API +------------------ + +Various functions are available for calling a Python object. +Each converts its arguments to a convention supported by the called object – +either *tp_call* or vectorcall. +In order to do as litle conversion as possible, pick one that best fits +the format of data you have available. + +The following table summarizes the available functions; +please see individual documentation for details. + ++------------------------------------------+------------------+--------------------+---------------+ +| Function | callable | args | kwargs | ++==========================================+==================+====================+===============+ +| :c:func:`PyObject_Call` | ``PyObject *`` | tuple | dict/``NULL`` | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallNoArgs` | ``PyObject *`` | --- | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallOneArg` | ``PyObject *`` | 1 object | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallObject` | ``PyObject *`` | tuple/``NULL`` | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallMethod` | obj + ``char*`` | format | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallFunctionObjArgs` | ``PyObject *`` | variadic | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallMethodObjArgs` | obj + ``char*`` | variadic | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_CallMethodOneArg` | obj + ``char*`` | 1 object | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_Vectorcall` | ``PyObject *`` | vectorcall | vectorcall | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_FastCallDict` | ``PyObject *`` | array & size | dict/``NULL`` | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_VectorcallMethod` | arg + ``char*`` | vectorcall | vectorcall | ++------------------------------------------+------------------+--------------------+---------------+ .. c:function:: PyObject* PyObject_Call(PyObject *callable, PyObject *args, PyObject *kwargs) @@ -151,6 +220,28 @@ C API Functions ``callable(*args, **kwargs)``. +.. c:function:: PyObject* PyObject_CallNoArgs(PyObject *callable) + + Call a callable Python object *callable* without any arguments. It is the + most efficient way to call a callable Python object without any argument. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + +.. c:function:: PyObject* _PyObject_CallOneArg(PyObject *callable, PyObject *arg) + + Call a callable Python object *callable* with exactly 1 positional argument + *arg* and no keyword arguments. + + Return the result of the call on success, or raise an exception and return + *NULL* on failure. + + .. versionadded:: 3.9 + + .. c:function:: PyObject* PyObject_CallObject(PyObject *callable, PyObject *args) Call a callable Python object *callable*, with arguments given by the @@ -286,43 +377,6 @@ C API Functions .. versionadded:: 3.8 -.. c:function:: Py_ssize_t PyVectorcall_NARGS(size_t nargsf) - - Given a vectorcall *nargsf* argument, return the actual number of - arguments. - Currently equivalent to:: - - (Py_ssize_t)(nargsf & ~PY_VECTORCALL_ARGUMENTS_OFFSET) - - However, the function ``PyVectorcall_NARGS`` should be used to allow - for future extensions. - - .. versionadded:: 3.8 - -.. c:function:: vectorcallfunc _PyVectorcall_Function(PyObject *op) - - If *op* does not support the vectorcall protocol (either because the type - does not or because the specific instance does not), return *NULL*. - Otherwise, return the vectorcall function pointer stored in *op*. - This function never raises an exception. - - This is mostly useful to check whether or not *op* supports vectorcall, - which can be done by checking ``_PyVectorcall_Function(op) != NULL``. - - .. versionadded:: 3.8 - -.. c:function:: PyObject* PyVectorcall_Call(PyObject *callable, PyObject *tuple, PyObject *dict) - - Call *callable*'s :c:type:`vectorcallfunc` with positional and keyword - arguments given in a tuple and dict, respectively. - - This is a specialized function, intended to be put in the - :c:member:`~PyTypeObject.tp_call` slot or be used in an implementation of ``tp_call``. - It does not check the :const:`_Py_TPFLAGS_HAVE_VECTORCALL` flag - and it does not fall back to ``tp_call``. - - .. versionadded:: 3.8 - .. c:function:: PyObject* _PyObject_VectorcallMethod(PyObject *name, PyObject *const *args, size_t nargsf, PyObject *kwnames) Call a method using the vectorcall calling convention. The name of the method @@ -335,10 +389,19 @@ C API Functions :c:func:`_PyObject_Vectorcall`. If the object has the :const:`Py_TPFLAGS_METHOD_DESCRIPTOR` feature, - this will actually call the unbound method object with the full + this will call the unbound method object with the full *args* vector as arguments. Return the result of the call on success, or raise an exception and return *NULL* on failure. .. versionadded:: 3.9 + + +Call Support API +---------------- + +.. c:function:: int PyCallable_Check(PyObject *o) + + Determine if the object *o* is callable. Return ``1`` if the object is callable + and ``0`` otherwise. This function always succeeds. From 28ba66cbfbcc72c8d5346378ac2d55e98c54cff0 Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 12 Sep 2019 10:36:02 +0100 Subject: [PATCH 5/5] Address review comments - Adjust vectorcall warning to not talk about a single function - Add underscore to _PyObject_CallOneArg - Include PyObject_CallFunction and _PyObject_CallMethodNoArgs - Summarize _PyObject_FastCallDict's args handling as "vectorcall" - Use "name" rather chan "char*" when the name is a Python object --- Doc/c-api/call.rst | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/Doc/c-api/call.rst b/Doc/c-api/call.rst index 74aa2d3803bc60..0833531b1d5ee1 100644 --- a/Doc/c-api/call.rst +++ b/Doc/c-api/call.rst @@ -43,8 +43,8 @@ for making calls more efficient. .. warning:: The vectorcall API is provisional and expected to become public in - Python 3.9, with a different name and, possibly, changed semantics. - If you use the function, plan for updating your code for Python 3.9. + Python 3.9, with a different names and, possibly, changed semantics. + If you use the it, plan for updating your code for Python 3.9. As rule of thumb, CPython will prefer the vectorcall for internal calls if the callable supports it. However, this is not a hard rule. @@ -185,23 +185,27 @@ please see individual documentation for details. +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallNoArgs` | ``PyObject *`` | --- | --- | +------------------------------------------+------------------+--------------------+---------------+ -| :c:func:`PyObject_CallOneArg` | ``PyObject *`` | 1 object | --- | +| :c:func:`_PyObject_CallOneArg` | ``PyObject *`` | 1 object | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallObject` | ``PyObject *`` | tuple/``NULL`` | --- | +------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`PyObject_CallFunction` | ``PyObject *`` | format | --- | ++------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallMethod` | obj + ``char*`` | format | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`PyObject_CallFunctionObjArgs` | ``PyObject *`` | variadic | --- | +------------------------------------------+------------------+--------------------+---------------+ -| :c:func:`PyObject_CallMethodObjArgs` | obj + ``char*`` | variadic | --- | +| :c:func:`PyObject_CallMethodObjArgs` | obj + name | variadic | --- | ++------------------------------------------+------------------+--------------------+---------------+ +| :c:func:`_PyObject_CallMethodNoArgs` | obj + name | --- | --- | +------------------------------------------+------------------+--------------------+---------------+ -| :c:func:`_PyObject_CallMethodOneArg` | obj + ``char*`` | 1 object | --- | +| :c:func:`_PyObject_CallMethodOneArg` | obj + name | 1 object | --- | +------------------------------------------+------------------+--------------------+---------------+ | :c:func:`_PyObject_Vectorcall` | ``PyObject *`` | vectorcall | vectorcall | +------------------------------------------+------------------+--------------------+---------------+ -| :c:func:`_PyObject_FastCallDict` | ``PyObject *`` | array & size | dict/``NULL`` | +| :c:func:`_PyObject_FastCallDict` | ``PyObject *`` | vectorcall | dict/``NULL`` | +------------------------------------------+------------------+--------------------+---------------+ -| :c:func:`_PyObject_VectorcallMethod` | arg + ``char*`` | vectorcall | vectorcall | +| :c:func:`_PyObject_VectorcallMethod` | arg + name | vectorcall | vectorcall | +------------------------------------------+------------------+--------------------+---------------+