Skip to content

Pass opaque state to PyContext_AddWatcher callback #127124

Open
@rhansen

Description

@rhansen

Feature or enhancement

Proposal:

I would like to change the Context watcher C API (new to v3.14, see #119333) to pass an opaque pointer to the callback. Specifically, change:

/*
* Context object watcher callback function. The object passed to the callback
* is event-specific; see PyContextEvent for details.
*
* if the callback returns with an exception set, it must return -1. Otherwise
* it should return 0
*/
typedef int (*PyContext_WatchCallback)(PyContextEvent, PyObject *);
/*
* Register a per-interpreter callback that will be invoked for context object
* enter/exit events.
*
* Returns a handle that may be passed to PyContext_ClearWatcher on success,
* or -1 and sets and error if no more handles are available.
*/
PyAPI_FUNC(int) PyContext_AddWatcher(PyContext_WatchCallback callback);

to:

/*
 * Context object watcher callback function.  arg is the same pointer passed to
 * PyContext_AddWatcher when the callback was registered.  The object passed to
 * the callback is event-specific; see PyContextEvent for details.
 *
 * if the callback returns with an exception set, it must return -1. Otherwise
 * it should return 0
 */
typedef int PyContext_WatchCallback(
    void *arg, PyContextEvent event, PyObject *obj);

/*
 * Register a per-interpreter callback that will be invoked for context events.
 * arg is an optional opaque pointer that is passed back to the callback; it
 * can be used to manage state if desired.
 *
 * Returns a handle that may be passed to PyContext_ClearWatcher on success,
 * or -1 and sets and error if no more handles are available.
 */
PyAPI_FUNC(int) PyContext_AddWatcher(
    PyContext_WatchCallback *callback, void *arg);

The original idea was for the callback to get any required state from the current context, but:

  • The config/state is not guaranteed to be available in the context. For example, the Python code can do ctx = contextvars.Context() instead of ctx = contextvars.copy_context(). Or it can do copy_context(), but early during initialization and squirrel the context away for later use. The latter seems plausible for 3rd party libraries (where the user might not have much control).
  • The same watcher callback cannot be registered multiple times simultaneously, each with its own config/state. This came up while I was refactoring TestContextObjectWatchers to add some more tests for a change I’m working on.

I’m not sure how likely either is to come up in normal use, but I think it is worth changing the design before the 3.14 release cements it.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/v3-14a1-design-limitations-of-pycontext-addwatcher/68177/4

cc @fried

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)type-featureA feature request or enhancement

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions