gh-111489: Add PyList_FromArrayMoveRef() function

[vstinner]

Add new functions to create tuple and list from arrays:

• PyTuple_FromArray().
• PyTuple_FromArrayMoveRef().
• PyList_FromArrayMoveRef().

Add tests on new functions.

• Issue: [C API] Make _PyList_FromArraySteal() function public #111489

---

📚 Documentation preview 📚: https://cpython-previews--112975.org.readthedocs.build/

[vstinner]

I tried to clear array items (or set bytes to a pattern, such as PYMEM_DEADBYTE=0xDD) in PyTuple_FromArrayMoveRef() and PyList_FromArrayMoveRef() in debug mode, but it broke current code which expect the array to remain usable. Pseudo-code:

PyObject *const * args = &PyTuple_GET_ITEM(callargs, 0);
for (size_t i = 0; i < argcount; i++) {
    Py_INCREF(args[i]);
}

PyObject *u = _PyTuple_FromArraySteal(args, argcount);
// ... check if u == NULL ...

// ... code using callargs ...

So I chose to leave array unchanged.

[vstinner]

PyObject* PyList_FromArrayMoveRef(PyObject *const *array, Py_ssize_t size)

I would prefer to declare array type as PyObject **, but since the function does not modify the array, I chose to declare the type using const.

[sweeneyde]

I think it's confusing that the function consumes references when it succeeds but does not consume them when it fails. I would think that it should always consume references or never consume references.

If it always consumes references, then one can write

    PyObject *pair[2];
    pair[0] = pair[1] = something;
    return _PyList_FromArraySteal(pair);

I think the only-sometimes-consuming was why there used to be a lot of error-handling mistakes in module initialization code with PyModule_AddObject. The docs note that

Unlike other functions that steal references, PyModule_AddObject() only releases the reference to value on success.

I would think that we would want to replicate the always-consuming "other functions" (e.g. PyTuple_SetItem) style, not the only-sometimes-consuming PyModule_AddObject style.

Is there a reason to prefer this style?

[serhiy-storchaka]

The Ref suffix is used in many new functions, would not it cause confusion?

What if use PyTuple_FromArrayMove() or PyTuple_FromArrayMoveOwnership()? Or maybe PyTuple_CopyFromArray() and PyTuple_MoveFromArray()?

Why there is no copying API for list?

[serhiy-storchaka]

Suggested change

	* Set an exception and return NULL on error.
	* Set an exception and return ``NULL`` on error.

[serhiy-storchaka]

I would use the following example:

Suggested change

	lst = [object() for _ in range(5)]
	lst = [[i] for _ in range(5)]

Better repr.

[vstinner]

First, I used list("abdef"), but all items are immortal and so tests didn't detect refcount bugs in my C code. I used object() to make sure that each object is unique.

I don't think that repr is important here, the test is not supposed to fail :-)

[serhiy-storchaka]

Test also with zero and negative size. with too large size (causing MemoryError or OverflowError), with array=NULL, and with NULLs in the array. We should know what cases raise exceptions, what cases crash and what cases are valid, and how errors are handled.

[serhiy-storchaka]

It does not match the documentation.

[vstinner]

What if use PyTuple_FromArrayMove() or PyTuple_FromArrayMoveOwnership()? Or maybe PyTuple_CopyFromArray() and PyTuple_MoveFromArray()?

I suggest discussing the name at: capi-workgroup/api-evolution#25

[vstinner]

Why there is no copying API for list?

I only exposed what already existed. But yeah, when I wrote the PR, I also noticed that it seems like there is "a hole" in the API :-) I can add a "copy" API for list as well.

[vstinner]

I convert this PR to a draft until we agree on naming: capi-workgroup/api-evolution#25

[vstinner]

I convert this PR to a draft until we agree on naming: capi-workgroup/api-evolution#25

I created this PR to have a more concrete API to discuss the naming.

I prefer to close the PR for now, until we agree on naming.