Terminology for managing ownership

[encukou]

Per capi-workgroup/problems#11, we need terms for managing ownership.
IMO, it would be useful to use similar terms both for references and to non-refcounted resources -- typically, allocated memory.

Arguments

By default, ownership of arguments is not transferred. After a call:

• the caller is still responsible for decrefing/freeing any arguments it passed in.
• the callee must not assume the argument stays valid.

The caller must guarantee that the arguments stay valid for the duration of the call.

“Output” arguments (where the callee gets a *result pointer and fills it) follow rules for return values.

We should use shorter, context-specific terms for transferring ownership (the non-default-behaviour):

• Steal: take ownership of PyObject* or other refcounted/GCd data (i.e. other references might exist). Used for performance optimizations or ergonomics.
• Consume: used for non-refcounted data (e.g. array of PyObject*), e.g. string builder finalization API would consume the scratch data and return a new object
• Clear: Destroy a reference and mark it invalid (i.e. set the PyObject* to NULL)
  • conflicts with the term for emptying a container
• Decref, Close, Free, Dealloc: specific API that only destroys the reference/data

I'm aware that Steal is pejorative; but IMO it's better than the alternative (Take, Give, Move)

Another useful exception is:

• Static: The argument is static and immutable; it does not need to be freed and is always valid.

Return values

By default, ownership of the return value is transferred. After a call:

• the caller is responsible for decrefing/freeing the return value.
• the callee must not assume the return value stays valid

The opposite is:

• Borrow: the caller must not decref/free the return value, but is responsible for ensuring that it is not accessed after its lifetime ends. Docs for the called function need to indicate what that lifetime is.

• Static is, conceptually, borrowing from the interpreter itself.

[davidhewitt]

I wonder, even though there is already use of Steal, does there need to be a difference for non-refcounted data? Why not use Consume for both types of data?

[encukou]

I wonder, even though there is already use of Steal, does there need to be a difference for non-refcounted data? Why not use Consume for both types of data?

To me, “consume” sounds like it'll destroy the object, not just the reference to it.

[vstinner]

I dislike "steal" term, it's negative and I don't think that it's easy to get its meaning.

I prefer "move" or "transfer". In python/cpython#111489 I propose to make _PyList_FromArraySteal() public. I would prefer to replace the "Steal" suffix with "Transfer" or "Move". It would be more explicit that the ownership is moved/transfered from somewhere to somewhere else. It describes better the action and that the old place becomes invalid.

[vstinner]

I wrote python/cpython#112975 which adds fuctions:

• PyTuple_FromArray()
• PyTuple_FromArrayMoveRef()
• PyList_FromArrayMoveRef()

I chose MoveRef suffix, similar to Ref in PyDict_GetItemRef, and using verb + Ref such as Py_SETREF().

Maybe with a concrete PR, it may be easier to decide on the naming convention for functions which move/transfer reference ownership.

The current implementation leaves the source array unchanged.

[gvanrossum]

FWIW I still prefer “steal”. So this is not settled.

[vstinner]

I chose MoveRef suffix, similar to Ref in PyDict_GetItemRef, and using verb + Ref such as Py_SETREF().

@serhiy-storchaka wrote:

The Ref suffix is used in many new functions, would not it cause confusion?

What if use PyTuple_FromArrayMove() or PyTuple_FromArrayMoveOwnership()? Or maybe PyTuple_CopyFromArray() and PyTuple_MoveFromArray()?

[encukou]

IMO, the proper term, which should be used in documentation, is transfer ownership. However, that's too long for an API name suffix. Using just Move would be unclear: what is being moved?
Steal is short and unique. I, unlike Victor, think it is easy to get its meaning.

[vstinner]

@gvanrossum:

FWIW I still prefer “steal”. So this is not settled.

Creating a PR doesn't imply that anything is settled. As I wrote, I proposed a PR to help to better see which name fits better.

[vstinner]

In terms of "ownership", Rust has many concepts and tools for that, and ownership is a key concept in Rust. In Rust, the action of transferring ownership is called "move": https://doc.rust-lang.org/rust-by-example/scope/move.html

When doing assignments (let x = y) or passing function arguments by value (foo(x)), the ownership of the resources is transferred. In Rust-speak, this is known as a move.

C++11 introduced std::move(obj) which is similar similar: https://en.cppreference.com/w/cpp/utility/move

std::move is used to indicate that an object t may be "moved from", i.e. allowing the efficient transfer of resources from t to another object.