Tracking issue for thread_local stabilization

[aturon]

The #[thread_local] attribute is currently feature-gated. This issue tracks its stabilization.

Known problems:

• #[thread_local] translates directly to the thread_local attribute in LLVM. This isn't supported on all platforms, and it's not even supported on all distributions within the same platform (e.g. macOS 10.6 didn't support it but 10.7 does). I don't think this is necessarily a blocker, but I also don't think we have many attributes and such which are so platform specific like this.
• Statics that are thread local shouldn't require Sync - #[thread_local] statics shouldn't require Sync #18001
• Statics that are thread local should either not borrow for the 'static lifetime or should be unsafe to access - Safe access to a #[thread_local] should be disallowed #17954
• Statics can currently reference other thread local statics, but this is a bug - Statics should not be able to take the address of a thread_local static #18712
• Unsound with generators Keeping references to #[thread_local] statics is allowed across yields. #49682
• static mut can be given 'static lifetime with NLL (#[thread_local] static mut is allowed to have 'static lifetime #54366)

[alexcrichton]

A note, we've since implemented cfg(target_thread_local) which is in turn itself feature gated, but this may ease the "this isn't implemented on all platforms" worry.

[alexandermerritt]

Hi! Is there any update on the status? Nightly still requires statics to be Sync. I tried with:

rustc 1.13.0-nightly (acd3f796d 2016-08-28)
binary: rustc
commit-hash: acd3f796d26e9295db1eba1ef16e0d4cc3b96dd5
commit-date: 2016-08-28
host: x86_64-unknown-linux-gnu
release: 1.13.0-nightly

[mneumann]

@alexcrichton Any news on #[thread_local] becoming stabilized? AFAIK, at the moment it is impossible on DragonFly to access errno variable from stable code, other than directly from libstd. This blocks crates like nix on DragonFly, which want to access errno as well, but libstd is not exposing it, and stable code is not allowed to use feature(thread_local).

[alexcrichton]

@mneumann no, no progress. I'd recommend a C shim for now.

[mneumann]

@alexcrichton thanks. I am doing a shim now https://github.com/mneumann/errno-dragonfly-rs.

[real-felix]

The optimizations are too aggressive ;)

See this code:

#![feature(thread_local)]

#[thread_local]
pub static FOO: [&str; 1] = [ "Hello" ];

fn change_foo(s: &'static str) {
    FOO[0] = s;
}

fn main() {
    println!("{}", FOO[0]);
    change_foo("Test");
    println!("{}", FOO[0]);
}

The compiler does not detect the side effect in change_foo and removes the call in release. The output is:

Hello
Hello

[alexcrichton]

cc @eddyb, @Boiethios your example shouldn't actually compile because it should require static mut, not just static

[real-felix]

It compiles with the last nightly rustc.

[eddyb]

Oh, drat, this is from my shortening of the lifetime, i.e

rust/src/librustc/middle/mem_categorization.rs

Lines 657 to 662 in dead08c

	// `#[thread_local]` statics may not outlive the current function.
	for attr in &self.tcx.get_attrs(def_id)[..] {
	if attr.check_name("thread_local") {
	return Ok(self.cat_rvalue_node(id, span, expr_ty));
	}
	}

@nikomatsakis what should we do here? I want a static lvalue, with a non-'static lifetime.

[cramertj]

#18001, #17954 and #18712 were fixed by #43746.

@alexcrichton @eddyb Do you know any other blockers, or is this ready for stabilization?

[eddyb]

There's some emulation clang does IIRC, that we might want to do ourselves, to support #[thread_local] everywhere.

And there's #47053 which results from my initial attempt to limit references to thread-local statics to the function they were created in.

[alexcrichton]

@cramertj I've personally been under the impression that we're holding out on stabilizing this for as long as possible. We've stabilized very few (AFAIK) platform-specific attributes like this and I at least personally haven't ever looked to hard into stabilizing this.

One blocker (in my mind at least) is what @eddyb mentioned where this is a "portable" attribute yet LLVM has a bunch of emulation on targets that don't actually support it (I think MinGW is an example). I don't believe we should allow the attribute on such targets, but we'd have to do a lot of investigation to figure out those targets.

Is there motivation for stabilizing this though? That'd certainly provide some good motivation for digging out any remaining issues and looking at it very closely. I'd just personally been under the impression that there's little motivation to stabilize this other than it'd be a "nice to have" in situations here and there.

[Amanieu]

I am using #[thread_local] in my code in a no_std context: I allocate space for the TLS segment and set up the architecture TLS register to point to it. Therefore I think that it is important to expose the #[thread_local] attribute so that it can at least be used by low-level code.

The only thing that I'm not too happy about is that Sync is no longer required for #[thread_local]: this makes it more difficult to write signal-safe code, since all communication between signal handlers and the main thread should be done through atomic types.

[eddyb]

@Amanieu Signal/interrupt-safe code has other requirements which are not satisfied by current Rust.

[Amanieu]

@eddyb Not really, all you need to do is treat the signal handler as a separate thread of execution. The only requirement you need to enforce safety is that objects which are accessed by both the main thread and the signal handler must be Sync.

Of course you can still cause deadlocks if the main thread and signal handler both try to grab the same lock, but that's not a safety issue.

[dtolnay]

@cramertj regarding remaining blockers, I would consider the behavior that @Boiethios reported above needs to be fixed before stabilization. I filed #54901 to follow up.

[ChrisDenton]

I wonder if there's a path towards a minimal thread_local stabilization by being very restrictive?

E.g.:

• all access is by value only
• only allow Copy types (or maybe even restrict it to only a subset of primitives)
• don't allow lifetimes
• don't allow composing thread_local with other lang attributes (they may interact in "interesting" ways)
• make thread_local error if the platform does not support it. This would imply also stabilizing target_thread_local but maybe the name should be subject to bikeshedding first (e.g. has_static_thread_local) to be clear that it may still have runtime thread locals.

[RalfJung]

What is the motivation for that? Doesn't thread_local! { static NAME = const { ... } } suffice?

[ChrisDenton]

no_std mainly

[RalfJung]

Does that form of the macro need anything that requires std? Could there be a core::thread_local! { ... } macro that makes that part of the functionality available without std (i.e., it would require const blocks)?

[ChrisDenton]

We could make a macro I guess. It'd be pretty redundant though when/if thread_local proper is stabilized. Unless we decide to just have a macro and not the attribute.

[RalfJung]

The API provided by #[thread_local] static and const-thread_local is pretty different I think. So your proposal does introduce redundancy that we currently don't have. There'd be two stable ways to do the same thing and two completely different mechanisms to ensure the necessary restrictions (such as "no 'static references").

[ChrisDenton]

The "two ways to do things" will occur whatever happens unless we simply don't stabilize #[thread_local] static.

[RalfJung]

That is true. If the usecases are covered I don't see a reason to stabilize #[thread_local] static. It could be transitioned to an internal feature, an implementation detail of the public macros.

[ChrisDenton]

core::thread_local! would also need LocalKey moved into core::thread (or an equivalent to it). One thing that thread_local! forces that #[thread_local] doesn't is going through a shared reference. There would also still need to be a way to know if thread_local is supported in no_std.

[newpavlov]

What is the motivation for that? Doesn't thread_local! { static NAME = const { ... } } suffice?

IIRC generated assembly for thread_local! was quite ugly when compared to equivalent #[thread_local] code. I haven't measured performance impact and do not know if it's possible to work around it with std changes, but it's still an example of non-zero-costness. Also, #[thread_local]-based code simply looks nicer and more ergonomic.

[ChrisDenton]

For the new thread_local! { static FOO: ... = const { ... } syntax it should all be optimized down to a minimum. If not then we should really try to fix that.

EDIT: assuming the type of FOO has no drop, of course.

[RalfJung]

Also, #[thread_local]-based code simply looks nicer and more ergonomic.

If they only work for Copy types and you can't take any references (and hence also not call any &self/&mut self methods), I am not sure if that's still true.

And taking references would be unsound.

[bjorn3]

And taking references would be unsound.

Borrowck limits those references to the current function already I believe.

[ChrisDenton]

Yes. The goal would be to eventually make full #[thread_local] stabilized (once bugs, etc are fixed). But that's not happening any time soon and in the meantime a minimal stabilization would be useful.

[RalfJung]

Oh interesting, I wasn't aware that borrowck had special treatment for thread_local statics.

[chorman0773]

I'd be very interested in seeing this stablized.
I use #[thread_local] in the interface code for my WIP OS, as an import/export for thread_local statics (https://github.com/LiliumOS/lilium-sys/blob/main/src/sys/io.rs#L44-L54). The handles here are thread-local (as handles themselves are thread-local resources in the OS) that are initialized by the USI (userspace standard interface) when you create a thread (This specific case is limited to a Copy type and you probably won't be be borrowing the handles often at all).

In general, being able to import an external TLS var without shim code written in C is useful (for example, if you want to grab __errno). This cannot be done with LocalKey and std::thread_local!.
#[thread_local] would also enable using TLS in a no-std context, such as a kernel.

[newpavlov]

@RalfJung

If they only work for Copy types and you can't take any references (and hence also not call any &self/&mut self methods), I am not sure if that's still true.

How about introducing a pointer-based variant of #[thread_local]? Something like this:

// Creates TLS value which stores 42 and creates "pointer" `FOO` to it.
#[thread_local_ptr]
static FOO: *mut u64 = 42u64;

fn increment_foo() {
    // SAFETY: reference to `FOO`does not escape execution thread
    let foo: &mut u64  = unsafe { &mut *FOO };
    *foo += 1;
}

[RalfJung]

@newpavlov it seems someone already taught the borrow checker about thread_local so taking references to them is actually sound. That's pretty cool.

I don't like there being this API duplication and inconsistency between that and the thread_local! macro, but that's up to libs-api to figure out.

We'd have to be rather careful where we enable this feature; in the past, I think on some Windows targets we switched back-and-forth between "true" thread-local statics and some other implementation for the thread_local! macro. The macro gives us the flexibility to do that; #[thread_local] does not, so once we allow it somewhere, if we later figure out there is some platform issue then we are in trouble.

I'm slightly bothered by thread-local statics pretending to be statics. They have very little in common with regular static in terms of their semantics. &FOO isn't even a constant, it is a function. We do reflect this in the MIR at least so bugs due to this are hopefully unlikely. And I don't have a proposal for a better syntax either so 🤷

[Amanieu]

I have a few use cases for #[thread_local] that can't be satisfied by the standard library's thread_local! macro:

• The code base is a no_std binary which doesn't depend on libc (it has its own TLS/stack initialization code).
• It exports #[thread_local] variables for use by C code (errno).
• It used to #[thread_local] variables in inline assembly (by symbol name), although that code has since been refactored and it not longer does that. It is possible to know the exact instruction sequence to use with the symbol because the whole code base is compiled with -Z tls-model=local-exec.

[dead-claudia]

#[thread_local] translates directly to the thread_local attribute in LLVM. This isn't supported on all platforms, and it's not even supported on all distributions within the same platform (e.g. 10.6 doesn't support it but 10.7 does). I don't think this is necessarily a blocker, but I also don't think we have many attributes and such which are so platform specific like this.

What's the status of this? I see a lot of discussion around a hypothetical cfg(target_thread_local), but nothing concrete?

[RalfJung]

It's not hypothetical, cfg(target_thread_local) exists on nightly. However, historically we have turned this on and off on some platforms to work around various bugs, so we should be careful before just blanket-exposing this on stable.

It used to #[thread_local] variables in inline assembly (by symbol name), although that code has since been refactored and it not longer does that. It is possible to know the exact instruction sequence to use with the symbol because the whole code base is compiled with -Z tls-model=local-exec.

If we just allow asm blocks to reference thread-locals, I worry it will lead to much confusion and errors. A thread-local is not just a normal symbol, after all -- but a Rust programmer might think it is, since in Rust code it behaves much like a static, but that's a sweet lie.

Unfortunately the inline asm docs don't even give an example for how sym is used at all. They do mention this though:

is allowed to point to a #[thread_local] static, in which case the asm code can combine the symbol with relocations (e.g. @plt, @tpoff) to read from thread-local data.

Presumably, it is UB to try to access that symbol without the exactly right set of relocations matching the current target and build flags? Seems like a pretty big footgun.

[Amanieu]

Presumably, it is UB to try to access that symbol without the exactly right set of relocations matching the current target and build flags? Seems like a pretty big footgun.

It's always safe to use the most general relocations (which involve calling __tls_get_addr). It's the more specific ones like @tpoff which are only valid with certain TLS models (for example local-exec is only valid in executables, not shared libraries).

[bjorn3]

Using __tls_get_addr requires you to emit a very specific set of bytes and relocations (on x86 this includes redundant prefixes). Getting anything wrong will likely cause tls relaxation by the linker to either error or generate a corrupt binary. And even with the most general relocations you still have to deal with different object file formats using different ways of accessing TLS. We currently don't have any cfg's for the object file format, so doing something that works correct on any OS for a given architecture is not possible.

[Qix-]

Just curious, are there any thread-local modes that aren't based on ELF's thread structure system? e.g. on x86_64 will we eventually be able to simply emit instructions that use FS/GS segments directly instead of reading a pointer from a negative offset and de-referencing it?

As far as I can tell that's GNU's/ELF's TLS model that doesn't really translate nicely to #![no_std] code (sans-OS) on x86_64.