-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tracking issue for cleaning up std's thread_local implementation details #110897
Comments
|
@joboet In what situation would that |
It's very cursed, but this works. I don't think it's currently tested, however. |
That just seems like a bug to me. A (thread local) |
Maybe 😄? It's sound however, as there can be no references to the data from the first initialization when the new value is written. |
Sure, but so would OnceCell be. ^^ And I think users should be able to safely assume that an immutable |
I did some software archeology and found the original issue and PR that introduced that mem::replace call: #30228 and #30267. The issue was originally about thread locals, but then most of the discussion was about drop on assignment in general. The issue description suggests:
Without a preference for any of these solutions. The PR also doesn't include a reason for picking one solution over the other. So I don't think the mem::replace behaviour was purposely picked over another solution (like dropping the other value or panicking). |
…-key, r=cuviper Remove unused std::sys_common::thread_local_key::Key Part of rust-lang#110897 This `Key` type seems unused. Let's remove it and see if anything explodes. :)
In this case I don't think anyone actually relies on LazyKeyInner being a OnceOrMaybeTwiceIfYouTryReallyHardCell instead of just a OnceCell. ^^ |
I'd advocate for a crater run, just in case, but yes. |
It is extremely unlikely someone is going to have tests that stress this exact point sufficiently for it to be revealed. |
Ok, yes, it's actually part of the documentation. Nevermind, then. |
Thanks for this, I've been very unhappy with the
There are a few others I had written down (multiple implementations of the manual tls-dtor list) but you touch on them and it might be outdated given #109858. |
…uviper Remove unused std::sys_common::thread_local_key::Key Part of rust-lang/rust#110897 This `Key` type seems unused. Let's remove it and see if anything explodes. :)
…n, r=dtolnay Broaden the consequences of recursive TLS initialization This PR updates the documentation of `LocalKey` to clearly disallow the behaviour described in [this comment](rust-lang#110897 (comment)). This allows using `OnceCell` for the lazy initialization of TLS variables, which panics on reentrant initialization instead of updating the value like TLS variables currently do. `@rustbot` label +T-libs-api r? `@m-ou-se`
…n, r=dtolnay Broaden the consequences of recursive TLS initialization This PR updates the documentation of `LocalKey` to clearly disallow the behaviour described in [this comment](rust-lang#110897 (comment)). This allows using `OnceCell` for the lazy initialization of TLS variables, which panics on reentrant initialization instead of updating the value like TLS variables currently do. ``@rustbot`` label +T-libs-api r? ``@m-ou-se``
Rollup merge of rust-lang#116172 - joboet:recursive_tls_initialization, r=dtolnay Broaden the consequences of recursive TLS initialization This PR updates the documentation of `LocalKey` to clearly disallow the behaviour described in [this comment](rust-lang#110897 (comment)). This allows using `OnceCell` for the lazy initialization of TLS variables, which panics on reentrant initialization instead of updating the value like TLS variables currently do. ``@rustbot`` label +T-libs-api r? ``@m-ou-se``
Rewrite native thread-local storage (part of rust-lang#110897) The current native thread-local storage implementation has become quite messy, uses indescriptive names and unnecessarily adds code to the macro expansion. This PR tries to fix that by using a new implementation that also allows more layout optimizations and potentially increases performance by eliminating unnecessary TLS accesses. This does not change the recursive initialization behaviour I described in [this comment](rust-lang#110897 (comment)), so it should be a library-only change. Changing that behaviour should be quite easy now, however. r? `@m-ou-se` `@rustbot` label +T-libs
Rewrite native thread-local storage (part of rust-lang#110897) The current native thread-local storage implementation has become quite messy, uses indescriptive names and unnecessarily adds code to the macro expansion. This PR tries to fix that by using a new implementation that also allows more layout optimizations and potentially increases performance by eliminating unnecessary TLS accesses. This does not change the recursive initialization behaviour I described in [this comment](rust-lang#110897 (comment)), so it should be a library-only change. Changing that behaviour should be quite easy now, however. r? `@m-ou-se` `@rustbot` label +T-libs
Rewrite native thread-local storage (part of rust-lang#110897) The current native thread-local storage implementation has become quite messy, uses indescriptive names and unnecessarily adds code to the macro expansion. This PR tries to fix that by using a new implementation that also allows more layout optimizations and potentially increases performance by eliminating unnecessary TLS accesses. This does not change the recursive initialization behaviour I described in [this comment](rust-lang#110897 (comment)), so it should be a library-only change. Changing that behaviour should be quite easy now, however. r? `@m-ou-se` `@rustbot` label +T-libs
Simplify key-based thread locals This PR simplifies key-based thread-locals by: * unifying the macro expansion of `const` and non-`const` initializers * reducing the amount of code in the expansion * simply reallocating on recursive initialization instead of going through `LazyKeyInner` * replacing `catch_unwind` with the shared `abort_on_dtor_unwind` It does not change the initialization behaviour described in rust-lang#110897.
Rewrite native thread-local storage (part of rust-lang#110897) The current native thread-local storage implementation has become quite messy, uses indescriptive names and unnecessarily adds code to the macro expansion. This PR tries to fix that by using a new implementation that also allows more layout optimizations and potentially increases performance by eliminating unnecessary TLS accesses. This does not change the recursive initialization behaviour I described in [this comment](rust-lang#110897 (comment)), so it should be a library-only change. Changing that behaviour should be quite easy now, however. r? `@m-ou-se` `@rustbot` label +T-libs
Rewrite TLS on platforms without threads The saga of rust-lang#110897 continues! r? `@m-ou-se` if you have time
Rewrite native thread-local storage (part of #110897) The current native thread-local storage implementation has become quite messy, uses indescriptive names and unnecessarily adds code to the macro expansion. This PR tries to fix that by using a new implementation that also allows more layout optimizations and potentially increases performance by eliminating unnecessary TLS accesses. This does not change the recursive initialization behaviour I described in [this comment](rust-lang/rust#110897 (comment)), so it should be a library-only change. Changing that behaviour should be quite easy now, however. r? `@m-ou-se` `@rustbot` label +T-libs
Simplify key-based thread locals This PR simplifies key-based thread-locals by: * unifying the macro expansion of `const` and non-`const` initializers * reducing the amount of code in the expansion * simply reallocating on recursive initialization instead of going through `LazyKeyInner` * replacing `catch_unwind` with the shared `abort_on_dtor_unwind` It does not change the initialization behaviour described in rust-lang#110897.
Rewrite native thread-local storage (part of #110897) The current native thread-local storage implementation has become quite messy, uses indescriptive names and unnecessarily adds code to the macro expansion. This PR tries to fix that by using a new implementation that also allows more layout optimizations and potentially increases performance by eliminating unnecessary TLS accesses. This does not change the recursive initialization behaviour I described in [this comment](rust-lang/rust#110897 (comment)), so it should be a library-only change. Changing that behaviour should be quite easy now, however. r? `@m-ou-se` `@rustbot` label +T-libs
As discovered by Mara in rust-lang#110897, our TLS implementation is a total mess. In the past months, I have simplified the actual macros and their expansions, but the majority of the complexity comes from the platform-specific support code needed to create keys and register destructors. In keeping with rust-lang#117276, I have therefore moved all of the `thread_local_key`/`thread_local_dtor` functions to the `thread_local` module in `sys` and given them a new structure, so that future porters of `std` can simply mix-and-match the existing code instead of having to copy the same (bad) implementation everywhere. The new structure should become obvious when looking at `sys/thread_local/mod.rs`. Unfortunately, the documentation changes associated with the refactoring have made this PR rather large. That said, this contains no functional changes except for two small ones: * the key-based destructor fallback now, by virtue of sharing the implementation used by macOS and others, stores its list in a `#[thread_local]` static instead of in the key, eliminating one indirection layer and drastically simplifying its code. * I've switched over ZKVM (tier 3) to use the same implementation as WebAssembly, as the implementation was just a way worse version of that Please let me know if I can make this easier to review! I know these large PRs aren't optimal, but I couldn't think of any good intermediate steps. @rustbot label +A-thread-locals
As discovered by Mara in rust-lang#110897, our TLS implementation is a total mess. In the past months, I have simplified the actual macros and their expansions, but the majority of the complexity comes from the platform-specific support code needed to create keys and register destructors. In keeping with rust-lang#117276, I have therefore moved all of the `thread_local_key`/`thread_local_dtor` modules to the `thread_local` module in `sys` and merged them into a new structure, so that future porters of `std` can simply mix-and-match the existing code instead of having to copy the same (bad) implementation everywhere. The new structure should become obvious when looking at `sys/thread_local/mod.rs`. Unfortunately, the documentation changes associated with the refactoring have made this PR rather large. That said, this contains no functional changes except for two small ones: * the key-based destructor fallback now, by virtue of sharing the implementation used by macOS and others, stores its list in a `#[thread_local]` static instead of in the key, eliminating one indirection layer and drastically simplifying its code. * I've switched over ZKVM (tier 3) to use the same implementation as WebAssembly, as the implementation was just a way worse version of that Please let me know if I can make this easier to review! I know these large PRs aren't optimal, but I couldn't think of any good intermediate steps. @rustbot label +A-thread-locals
… r=Mark-Simulacrum std: refactor the TLS implementation As discovered by Mara in rust-lang#110897, our TLS implementation is a total mess. In the past months, I have simplified the actual macros and their expansions, but the majority of the complexity comes from the platform-specific support code needed to create keys and register destructors. In keeping with rust-lang#117276, I have therefore moved all of the `thread_local_key`/`thread_local_dtor` modules to the `thread_local` module in `sys` and merged them into a new structure, so that future porters of `std` can simply mix-and-match the existing code instead of having to copy the same (bad) implementation everywhere. The new structure should become obvious when looking at `sys/thread_local/mod.rs`. Unfortunately, the documentation changes associated with the refactoring have made this PR rather large. That said, this contains no functional changes except for two small ones: * the key-based destructor fallback now, by virtue of sharing the implementation used by macOS and others, stores its list in a `#[thread_local]` static instead of in the key, eliminating one indirection layer and drastically simplifying its code. * I've switched over ZKVM (tier 3) to use the same implementation as WebAssembly, as the implementation was just a way worse version of that Please let me know if I can make this easier to review! I know these large PRs aren't optimal, but I couldn't think of any good intermediate steps. `@rustbot` label +A-thread-locals
…=Mark-Simulacrum std: refactor the TLS implementation As discovered by Mara in rust-lang#110897, our TLS implementation is a total mess. In the past months, I have simplified the actual macros and their expansions, but the majority of the complexity comes from the platform-specific support code needed to create keys and register destructors. In keeping with rust-lang#117276, I have therefore moved all of the `thread_local_key`/`thread_local_dtor` modules to the `thread_local` module in `sys` and merged them into a new structure, so that future porters of `std` can simply mix-and-match the existing code instead of having to copy the same (bad) implementation everywhere. The new structure should become obvious when looking at `sys/thread_local/mod.rs`. Unfortunately, the documentation changes associated with the refactoring have made this PR rather large. That said, this contains no functional changes except for two small ones: * the key-based destructor fallback now, by virtue of sharing the implementation used by macOS and others, stores its list in a `#[thread_local]` static instead of in the key, eliminating one indirection layer and drastically simplifying its code. * I've switched over ZKVM (tier 3) to use the same implementation as WebAssembly, as the implementation was just a way worse version of that Please let me know if I can make this easier to review! I know these large PRs aren't optimal, but I couldn't think of any good intermediate steps. `@rustbot` label +A-thread-locals
…=Mark-Simulacrum std: refactor the TLS implementation As discovered by Mara in rust-lang#110897, our TLS implementation is a total mess. In the past months, I have simplified the actual macros and their expansions, but the majority of the complexity comes from the platform-specific support code needed to create keys and register destructors. In keeping with rust-lang#117276, I have therefore moved all of the `thread_local_key`/`thread_local_dtor` modules to the `thread_local` module in `sys` and merged them into a new structure, so that future porters of `std` can simply mix-and-match the existing code instead of having to copy the same (bad) implementation everywhere. The new structure should become obvious when looking at `sys/thread_local/mod.rs`. Unfortunately, the documentation changes associated with the refactoring have made this PR rather large. That said, this contains no functional changes except for two small ones: * the key-based destructor fallback now, by virtue of sharing the implementation used by macOS and others, stores its list in a `#[thread_local]` static instead of in the key, eliminating one indirection layer and drastically simplifying its code. * I've switched over ZKVM (tier 3) to use the same implementation as WebAssembly, as the implementation was just a way worse version of that Please let me know if I can make this easier to review! I know these large PRs aren't optimal, but I couldn't think of any good intermediate steps. `@rustbot` label +A-thread-locals
This might have caused #126992 |
std: separate TLS key creation from TLS access Currently, `std` performs an atomic load to get the OS key on every access to `StaticKey` even when the key is already known. This PR thus replaces `StaticKey` with the platform-specific `get` and `set` function and a new `LazyKey` type that acts as a `LazyLock<Key>`, allowing the reuse of the retreived key for multiple accesses. Related to rust-lang#110897.
Rollup merge of rust-lang#126953 - joboet:lazy_key, r=jhpratt std: separate TLS key creation from TLS access Currently, `std` performs an atomic load to get the OS key on every access to `StaticKey` even when the key is already known. This PR thus replaces `StaticKey` with the platform-specific `get` and `set` function and a new `LazyKey` type that acts as a `LazyLock<Key>`, allowing the reuse of the retreived key for multiple accesses. Related to rust-lang#110897.
std::thread_local
,std::thread::local
,std::thread::local_impl
,std::sys_common::thread_local_key
,std::sys_common::thread_local_dtor
,std::sys::thread_local_key
, etc. etc. are all messy and form quite a confusing maze. Just look at this map I tried to draw of it all:AAAaaaaAaa
Time to clean it up.
And maybe also fix some bugs while we're at it. ^^
Key
type: Remove unused std::sys_common::thread_local_key::Key #110898extern "C"
inthread_local::fast_local
#112292Key
, because most of those things are actually not keys, and dropLazyKeyInner
thread_local!{}
expansion. Make TLS accessors closures that return pointers #125525*const T
rather than&'static T
when it's not really'static
: Make TLS accessors closures that return pointers #125525#[thread_local]
)?ReentrantLock
#124881thread::current
thread local more efficient and make it available in (most) TLS destructors: std: makethread::current
available in allthread_local!
destructors #127912Related issues to solve, maybe:
Stack overflow in thread local's drop rendered as a segmentation fault. #109785The text was updated successfully, but these errors were encountered: