Stabilize target_feature_11

[veluca93]

Stabilization report

This is an updated version of #116114, which is itself a redo of #99767. Most of this commit and report were copied from those PRs. Thanks @LeSeulArtichaut and @calebzulawski!

Summary

Allows for safe functions to be marked with #[target_feature] attributes.

Functions marked with #[target_feature] are generally considered as unsafe functions: they are unsafe to call, cannot be assigned to safe function pointers, and don't implement the Fn* traits.

However, calling them from other #[target_feature] functions with a superset of features is safe.

// Demonstration function
#[target_feature(enable = "avx2")]
fn avx2() {}

fn foo() {
    // Calling `avx2` here is unsafe, as we must ensure
    // that AVX is available first.
    unsafe {
        avx2();
    }
}

#[target_feature(enable = "avx2")]
fn bar() {
    // Calling `avx2` here is safe.
    avx2();
}

Test cases

Tests for this feature can be found in tests/ui/target_feature/.

Edge cases

Closures

• target-feature 1.1: should closures inherit target-feature annotations? #73631

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

#[target_feature(enable = "avx2")]
fn qux() {
    let my_closure = || avx2(); // this call to `avx2` is safe
    let f: fn() = my_closure;
}

This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function, and any closures defined inside it, execute.

This is usually ensured because target features are assumed to never disappear, and:

• on any unsafe call to a #[target_feature] function, presence of the target feature is guaranteed by the programmer through the safety requirements of the unsafe call.
• on any safe call, this is guaranteed recursively by the caller.

If you work in an environment where target features can be disabled, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again).

Note: this has an effect on existing code, as nowadays closures do not inherit features from the enclosing function, and thus this strengthens a safety requirement. It was originally proposed in #73631 to solve this by adding a new type of UB: “taking a target feature away from your process after having run code that uses that target feature is UB” .
This was motivated by userspace code already assuming in a few places that CPU features never disappear from a program during execution (see i.e. https://github.com/rust-lang/stdarch/blob/2e29bdf90832931ea499755bb4ad7a6b0809295a/crates/std_detect/src/detect/arch/x86.rs); however, concerns were raised in the context of the Linux kernel; thus, we propose to relax that requirement to "causing the set of usable features to be reduced is unsafe; when doing so, the programmer is required to ensure that no closures or safe fn pointers that use removed features are still in scope".

• Fix #[inline(always)] on closures with target feature 1.1 #111836

Closures accept #[inline(always)], even within functions marked with #[target_feature]. Since these attributes conflict, #[inline(always)] wins out to maintain compatibility.

ABI concerns

• The extern "C" ABI of SIMD vector types depends on target features #116558

The ABI of some types can change when compiling a function with different target features. This could have introduced unsoundness with target_feature_11, but recent fixes (#133102, #132173) either make those situations invalid or make the ABI no longer dependent on features. Thus, those issues should no longer occur.

Special functions

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), safe default trait implementations and safe trait methods.

This was not disallowed at the time of the first stabilization PR for target_features_11, and resulted in the following issues/PRs:

• #[target_feature] is allowed on main #108645
• #[target_feature] is allowed on default implementations #108646
• #[target_feature] is allowed on #[panic_handler] with target_feature 1.1 #109411
• Prevent using #[target_feature] on lang item functions #115910

Documentation

• Reference: Document the target_feature_11 feature reference#1181

---

cc tracking issue #69098
cc @workingjubilee
cc @RalfJung
r? @rust-lang/lang

[workingjubilee]

This means that in order to call a function with #[target_feature], you must show that the target-feature is available while the function executes and for as long as whatever may escape from that function lives.

"whatever"?

[workingjubilee]

I dont think so, but our justification for why TF 1.1 is sound needs to be refined. That should be part of the updated stabilization report.

Ralf requested this on Zulip. I do not see that such a rejustification is included, and I agree that it should be. An assertion of "we do not expect problems from this" does not support itself.

[veluca93]

This means that in order to call a function with #[target_feature], you must show that the target-feature is available while the function executes and for as long as whatever may escape from that function lives.

"whatever"?

This is wording copied over from the previous PR - how would you reword it instead? Explicit list of things that might escape?

[veluca93]

I dont think so, but our justification for why TF 1.1 is sound needs to be refined. That should be part of the updated stabilization report.

Ralf requested this on Zulip. I do not see that such a rejustification is included, and I agree that it should be. An assertion of "we do not expect problems from this" does not support itself.

I updated the description around that sentence to include a more precise justification.

[RalfJung]

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

Oddly, this means this compiles

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    || avx2()
}

but this does not

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    avx2
}

That's not a blocker but might be worth an issue. (But changing this may mean we have to refine the safety condition.)

This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function executes and for as long as any returned closures (or function pointers) that inherit features from that function live.

The definition of when something "live"s is subtle. I would make it about execution: you must guarantee that the target feature is available while the function or any closure defined inside that function executes. This is generally ensured because target features, once available, cannot usually be taken back; if you work in an environment where they can be taken back, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again).

The #[target_feature] attribute is forbidden from a variety of special functions

There is an enumeration saying that the attribute is allowed on main etc, only to then state that it is forbidden. This is confusing.

Have we covered all special functions?

[veluca93]

Closures defined inside functions marked with #[target_feature] inherit the target features of their parent function. They can still be assigned to safe function pointers and implement the appropriate Fn* traits.

Oddly, this means this compiles

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    || avx2()
}

but this does not

#[target_feature(enable = "avx2")]
fn bar() -> fn() {
    avx2
}

That's not a blocker but might be worth an issue. (But changing this may mean we have to refine the safety condition.)

This is indeed somewhat odd; refining the safety condition to cover also converting functions-you-could-call to safe fn pointers ought to be a minimal change (extending from "closure" to also cover "safe fn pointers generated inside the function", or something like that), and I think I'd be OK with leaving this as a follow-up, but I do not have strong opinions.

This means that in order to call a function with #[target_feature], you must guarantee that the target-feature is available while the function executes and for as long as any returned closures (or function pointers) that inherit features from that function live.

The definition of when something "live"s is subtle. I would make it about execution: you must guarantee that the target feature is available while the function or any closure defined inside that function executes. This is generally ensured because target features, once available, cannot usually be taken back; if you work in an environment where they can be taken back, it is your responsibility to ensure that no code inside a target feature function (including inside a closure) runs after this (until the feature is enabled again).

The #[target_feature] attribute is forbidden from a variety of special functions

There is an enumeration saying that the attribute is allowed on main etc, only to then state that it is forbidden. This is confusing.

Have we covered all special functions?

I reworded both sections to hopefully increase clarity, PTAL.

[RalfJung]

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), and default trait implementations.

What about trait implementations (not the default ones, but the regular ones)?
What about the alloc error handler?
Is there some central system that makes it unlikely to forget checking for target feature the next time we add a special function like that?

[veluca93]

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), and default trait implementations.

What about trait implementations (not the default ones, but the regular ones)?

That is forbidden, I updated the first comment to reflect that.

What about the alloc error handler? Is there some central system that makes it unlikely to forget checking for target feature the next time we add a special function like that?

No to the first. I am not sure how such a central system could look like -- I assumed such special functions would be lang items, but either that is not true or the "current and future lang items" part of the stabilization report is a lie.

I will investigate -- but to begin with, I assume the alloc error handler should receive the same treatment as the other special functions.

[RalfJung]

I am not sure how such a central system could look like

At the very least we could have a shared helper checking all lang items, main, panic handler, alloc error handler -- it's not just target features we must be worried about but also #[track_caller]. Also see #124067; there is some shared checking for "weak lang items" but I don't know how widely used it is beyond that.

[veluca93]

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), and default trait implementations.

What about trait implementations (not the default ones, but the regular ones)?

That is forbidden, I updated the first comment to reflect that.

What about the alloc error handler? Is there some central system that makes it unlikely to forget checking for target feature the next time we add a special function like that?

No to the first. I am not sure how such a central system could look like -- I assumed such special functions would be lang items, but either that is not true or the "current and future lang items" part of the stabilization report is a lie.

I will investigate -- but to begin with, I assume the alloc error handler should receive the same treatment as the other special functions.

The central system appears to be here:

rust/compiler/rustc_passes/src/check_attr.rs

Line 719 in 8e37e15

// `#[target_feature]` is not allowed in lang items.

The issue that makes alloc_error_handler + target_feature not be an error is that alloc_error_handler is not considered a lang item, unlike panic_handler, by hir::lang_items::extract. That seems surprising to me, and probably also the cause of i.e. #83430 (since similar checks for track_caller also relies on extract recognizing functions as lang items).

So, the central system does appear to exist, and relies on keeping hir::lang_items::extract, which is relied upon in other parts of the compiler anyway.

[bors]

☔ The latest upstream changes (presumably #132706) made this pull request unmergeable. Please resolve the merge conflicts.

[RalfJung]

The #[target_feature] attribute is forbidden from a variety of special functions, such as main, current and future lang items (e.g. #[start], #[panic_handler]), default trait implementations and safe trait methods.

To be more precise, it is forbidden on safe default trait implementations. It is still allowed on unsafe fn here. This is all quite subtle.

[RalfJung]

@oli-obk @petrochenkov can you think of any other special functions that should not be allowed to have target features because they are invoked in places where we can't be sure that the features are available?

[oli-obk]

I think the current implementation is a footgun and we can never be sure we caught all such issues.

I think we should invert the logic: we allow you (at the syntax level) to declare safe target_feature functions, but the compiler just treats them as unsafe functions. At the few sites where they are safely callable (in unsafe blocks 😆 , in other target_feature functions of the same feature set, ...), we add checks to skip the unsafe requirement. This way we can remove the trait method check and don't have to think about other checks we need to add to make this sound. It's sound by default.