Skip to content

Commit

Permalink
gh-117953: Track Extra Details in Global Extensions Cache (gh-118532)
Browse files Browse the repository at this point in the history
We have only been tracking each module's PyModuleDef.  However, there are some problems with that.  For example, in some cases we load single-phase init extension modules from def->m_base.m_init or def->m_base.m_copy, but if multiple modules share a def then we can end up with unexpected behavior.

With this change, we track the following:

* PyModuleDef (same as before)
* for some modules, its init function or a copy of its __dict__, but specific to that module
* whether it is a builtin/core module or a "dynamic" extension
* the interpreter (ID) that owns the cached __dict__ (only if cached)

This also makes it easier to remember the module's kind (e.g. single-phase init) and if loading it previously failed, which I'm doing separately.
  • Loading branch information
ericsnowcurrently authored May 4, 2024
1 parent 978fba5 commit 291cfa4
Show file tree
Hide file tree
Showing 3 changed files with 578 additions and 138 deletions.
12 changes: 11 additions & 1 deletion Include/internal/pycore_importdl.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ typedef enum ext_module_kind {
_Py_ext_module_kind_INVALID = 3,
} _Py_ext_module_kind;

typedef enum ext_module_origin {
_Py_ext_module_origin_CORE = 1,
_Py_ext_module_origin_BUILTIN = 2,
_Py_ext_module_origin_DYNAMIC = 3,
} _Py_ext_module_origin;

/* Input for loading an extension module. */
struct _Py_ext_module_loader_info {
Expand All @@ -34,6 +39,7 @@ struct _Py_ext_module_loader_info {
/* path is always a borrowed ref of name or filename,
* depending on if it's builtin or not. */
PyObject *path;
_Py_ext_module_origin origin;
const char *hook_prefix;
const char *newcontext;
};
Expand All @@ -42,7 +48,11 @@ extern void _Py_ext_module_loader_info_clear(
extern int _Py_ext_module_loader_info_init(
struct _Py_ext_module_loader_info *info,
PyObject *name,
PyObject *filename);
PyObject *filename,
_Py_ext_module_origin origin);
extern int _Py_ext_module_loader_info_init_for_core(
struct _Py_ext_module_loader_info *p_info,
PyObject *name);
extern int _Py_ext_module_loader_info_init_for_builtin(
struct _Py_ext_module_loader_info *p_info,
PyObject *name);
Expand Down
Loading

0 comments on commit 291cfa4

Please sign in to comment.