Disable importing legacy single-phase init extensions within subinterpreters in --disable-gil build

[colesbury]

Feature or enhancement

When importing a single-phase init extension in a subinterpreter, Python will make a shallow copy of the module's dictionary, which can share (non-immortal) objects between interpreters.

This does not work properly in the --disable-gil build, and we should disable it by raising an ImportError, at least for now. We can investigate how to support this in the future.

There are currently some unit tests for this case. Those tests pass, but that's mostly because they are simple and small changes to things like the GC will cause them to crash.

The underlying problems are not directly related to the GIL, but rather because the GC and our mimalloc integration in the --disable-gil build assume that non-immortal objects are isolated by interpreter:

• The GC assumes that all tracked objects reachable via tp_traverse are also reachable from the per-interpreter mimalloc heaps. Violating this assumption can cause flags to be set to an inconsistent state.
• The mimalloc pool of abandoned segments is per-interpreter. If a non-immortal object outlives its creating interpreter, this can cause use-after-free problems.

Linked PRs

• gh-117649: Raise ImportError for unsupported modules in free-threaded build #117651
• gh-117649: Fix file descriptor leak in (expected) failing test case #117780

[ericsnowcurrently]

Note that this only relates to legacy subinterpreters (not isolated, created by Py_NewInterpreter()). With isolated interpreters single-phase init modules already trigger ImportError.

Raising ImportError under --disable-gil seems like a reasonable short-term solution.

[encukou]

You might want use the current mechanism: require PyInterpreterConfig.check_multi_interp_extensions to be 1 for Py_NewInterpreterFromConfig, and set it to 1 for Py_NewInterpreter (i.e. _PyInterpreterConfig_LEGACY_INIT).
And perhaps disable _imp._override_multi_interp_extensions_check.

Note that there are two ways to implement the limitation:

• allow the import only in the main interpreter
• allow the first import, regardless of what interp it's made in

The first was easier to implement; we might want to switch to the second later. If/when we do, --disable-gil builds might want to follow along.