-
-
Notifications
You must be signed in to change notification settings - Fork 31.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bpo-40512: Store pointer to interpreter state in a thread local variable #29228
bpo-40512: Store pointer to interpreter state in a thread local variable #29228
Conversation
1be2ad9
to
ada883d
Compare
🤖 New build scheduled with the buildbot fleet by @markshannon for commit ada883d6b97bb072ef8465b71671ef9db101c7ae 🤖 If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again. |
Build failures look like the usual suspects. |
ada883d
to
f22036b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks goot to me in general but it would be good if @vstinner could take a look since he has been modifying these APIs recently a lot.
@@ -7333,7 +7333,7 @@ os_forkpty_impl(PyObject *module) | |||
int master_fd = -1; | |||
pid_t pid; | |||
|
|||
if (_PyInterpreterState_GET() != PyInterpreterState_Main()) { | |||
if (PyInterpreterState_Get() != PyInterpreterState_Main()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (PyInterpreterState_Get() != PyInterpreterState_Main()) { | |
PyInterpreterState *interp = PyInterpreterState_Get(); | |
if (!_Py_IsMainInterpreter(interp)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is too easy to overlook the !
when making this change. Using the comparison operator makes it clearer that we are checking that interp
is not the main interpreter.
Both @ericsnowcurrently and I have made this mistake recently.
@vstinner Any comments? |
The old approach had a NULL check and |
Sorry, I don't have the bandwidth to review this change :-( |
@@ -40,6 +34,9 @@ _Py_GetMainConfig(void) | |||
return _PyInterpreterState_GetConfig(interp); | |||
} | |||
|
|||
#ifdef _Py_THREAD_LOCAL | |||
extern _Py_THREAD_LOCAL PyInterpreterState *_py_current_interpreter; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why the current interpreter and not the current PyThreadState
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two reasons.
- I though this would add the most value, as we need the interpreter state to get to freelists which are performance critical.
- We already pass the threadstate as an argument to a lot of functions, so I wanted to verify this approach before rewriting all that code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it make sense to do both? (They are both hot in different situations.) What is the cost to the number of thread-local variables?
@@ -509,7 +509,7 @@ _io_text_encoding_impl(PyObject *module, PyObject *encoding, int stacklevel) | |||
/*[clinic end generated code: output=91b2cfea6934cc0c input=bf70231213e2a7b4]*/ | |||
{ | |||
if (encoding == NULL || encoding == Py_None) { | |||
PyInterpreterState *interp = _PyInterpreterState_GET(); | |||
PyInterpreterState *interp = PyInterpreterState_Get(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why the switch to PyInterpreterState_Get()
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Modules cannot access a thread-local directly because they are dynamically loaded. It is a loader, rather a language, limitation, I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's good to know. Thanks for explaining. This might not be obvious to maintainers. Would you mind adding a short comment next to the declaration of _py_current_interpreter
noting the constraint?
Also, this doesn't affect builtin modules, right? Aren't some of the modules updated here builtin modules?
I'm dropping this as there are a few reasons why passing the interpreter state explicitly make sense.
|
A more future proof way to handle interpreter state. 1% faster as well.
https://bugs.python.org/issue40512