-
Notifications
You must be signed in to change notification settings - Fork 6
3.3 Interpreter Isolation
When it comes to the isolation of multiple interpreters in the same process, there are a number of factors to consider:
- interpreters typically execute in different threads, but may run in the same thread
- threads share a number of process-global resources
- threads share the process heap and the program's data and code segments
- each thread has its own stack, but other threads may reference memory there
- currently most resources shared by interpreters are protected by the GIL
- currently a process has at most one Python runtime under which all interpreter run
- ...
The following is a deeper look at the key top-level topics related to isolation between interpreters in the same process.
Threads
Tiers of Isolation in a Process
Tiers of Isolation in CPython
Isolation Gaps
Open Questions
In many ways, isolation between interpreters is an extension of isolation between threads. So here's a brief primer.
...
...
See:
...
"Apartment" - box of thread-specific data which can only be shared between threads explicitly
See:
Aside from Windows, there are a small number of operating systems (including even some *nix) that do not implement pthreads. However, either they are not supported in CPython, we handle them separately (Python/thread_*.h; none currently), or we (at least partially) simulate pthreads for them in Python/thread_pthread.h.
A Python thread is an interpreter-specific wrapper around an OS-level thread. The thread-specific data stored in PyThreadState, including the ID of the corresponding OS-level thread.
...
Interestingly, the Windows docs on threads seem to agreed with Python's historical philosophy on threading:
Using multiple threads is not a guarantee of better performance.
In fact, because thread factoring is a difficult problem, using
multiple threads often causes performance problems. The key is
to use multiple threads only if you are very sure of what you
are doing.
...
- (open) file descriptors
- environment variables
- signals
- ...
See: http://man7.org/linux/man-pages/man7/pthreads.7.html#DESCRIPTION
...
See: http://man7.org/linux/man-pages/man7/pthreads.7.html#DESCRIPTION
The Win32 docs makes a key observation:
The only global variables in the ... program are either
mutexes or variables that never change after they are initialized.
...
...
See: PyRuntimeState
...
See: PyInterpreterState
...
See: PyThreadState
- memory allocators / arenas (also have a global one?)
- C-level object caches
- freelists
- static types
- objects (cannot even be used in another interpreter?)
- modules and other interpreter-global state
...
- How are process global resources handled in other runtimes that support multiple runtimes in a single process?