You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of my recent PRs, gh-117662, led to one of the Windows refleak buildbots failing1. I'm fairly confident the PR revealed an existing source of refleaks (albeit an unlikely one), rather that introducing new leaks. I have resolved the failures with gh-117913, but the underlying potential source of refleaks remains.
I've been able to reproduce the leak conditions, but only on Windows (for now?). The following conditions are necessary:
interpreter created using a config with check_multi_interp_extensions=False (e.g. "legacy")
interpreter imports test.support.os_helper
try:
import_interpretersexceptModuleNotFoundError:
import_xxsubinterpretersas_interpretersconfig=_interpreters.new_config('legacy') # critically, check_multi_interp_extensions=Falseinterpid=_interpreters.create(config)
_interpreters.exec(interpid, 'import test.support.os_helper')
_interpreters.destroy(interpid)
# leaks a bunch of objects
When the bug is triggered, around 190 objects are leaked. At first I thought this was interpreter finalization failing silently, but now I think it is something else.
I suspect the underlying problem relates to legacy extension modules. In the specific example above, I'm pretty sure the _ctypes module is leaking and ~190 is how many objects it holds, directly or indirectly. The main clue there is that, when I run ./python -v ..., the _ctypes module is never noted as destroyed, whereas all other modules are. As to test.support.os_helper, _ctypes is one of the modules that gets indirectly imported. All that said, I haven't been able to reproduce the leak if the subinterpreter above imports _ctypes instead.
Why did we only see this on Windows? The only clue I can think of is that on Windows stdlib extension modules are all turned into builtin modules, IIRC. The logic of the check_multi_interp_extensions check (via _PyImport_CheckSubinterpIncompatibleExtensionAllowed() in Python/import.c) is specific to extension modules, not builtin modules.
The _ctypes module has switched to the multi-phase initialization, and I can confirm the leaks above disappear on Windows with a patch like #117181 (comment). I think we are on #117181 (comment) at this point.
Bug report
Bug description:
One of my recent PRs, gh-117662, led to one of the Windows refleak buildbots failing1. I'm fairly confident the PR revealed an existing source of refleaks (albeit an unlikely one), rather that introducing new leaks. I have resolved the failures with gh-117913, but the underlying potential source of refleaks remains.
I've been able to reproduce the leak conditions, but only on Windows (for now?). The following conditions are necessary:
check_multi_interp_extensions=False
(e.g. "legacy")When the bug is triggered, around 190 objects are leaked. At first I thought this was interpreter finalization failing silently, but now I think it is something else.
I suspect the underlying problem relates to legacy extension modules. In the specific example above, I'm pretty sure the
_ctypes
module is leaking and ~190 is how many objects it holds, directly or indirectly. The main clue there is that, when I run./python -v ...
, the_ctypes
module is never noted as destroyed, whereas all other modules are. As to test.support.os_helper,_ctypes
is one of the modules that gets indirectly imported. All that said, I haven't been able to reproduce the leak if the subinterpreter above imports_ctypes
instead.Why did we only see this on Windows? The only clue I can think of is that on Windows stdlib extension modules are all turned into builtin modules, IIRC. The logic of the
check_multi_interp_extensions
check (via_PyImport_CheckSubinterpIncompatibleExtensionAllowed()
in Python/import.c) is specific to extension modules, not builtin modules.CPython versions tested on:
CPython main branch
Operating systems tested on:
Windows
Footnotes
https://buildbot.python.org/all/#/builders/920/builds/733 ↩
The text was updated successfully, but these errors were encountered: