llvm.expect intrinsic from likely/unlikely is not emitted in release mode
#96276
Comments
|
Oops, @SUPERCILEX was faster :) Closing as a duplicate. |
|
Yours has more detail though so reposted. :) |
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Jan 22, 2024
#[cold] on match arms ### Edit This should be in T-lang. I'm not sure how I can change it. There is discussion: https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Allow.20.23.5Bcold.5D.20on.20match.20and.20if.20arms ### Summary Adds the possibility to use `#[cold]` attribute on match arms to hint the optimizer that the arm is unlikely to be taken. ### Motivation These hints are sometimes thought to help branch prediction, but the effect is probably marginal. Modern CPUs don't support hints on conditional branch instructions. They either have the current branch in the BTB (branch prediction buffer), or not, in which case the branch is predicted not to be taken. These hints are, however, helpful in letting the compiler know what is the fast path, so it can be optimized at the expense of the slow path. `grep`-ing the LLVM code for BlockFrequencyInfo and BranchProbabilityInfo shows that these hints are used at many places in the optimizer. Such as: - block placement - improve locality by making the fast path compact and move everything else out of the way - inlining, loop unrolling - these optimizations can be less aggressive on the cold path therefore reducing code size - register allocation - preferably keep in registers the data needed on the fast path ### History RFC 1131 ( rust-lang#26179 ) added `likely` and `unlikely` intrinsics, which get converted to `llvm.expect.i8`. However this LLVM instruction is fragile and may get removed by some optimization passes. The problems with the intrinsics have been reported several times: rust-lang#96276 , rust-lang#96275 , rust-lang#88767 ### Other languages Clang and GCC C++ compilers provide `__builtin_expect`. Since C++20, it is also possible to use `[[likely]]` and `[[unlikely]]` attributes. Use: ``` if (__builtin_expect(condition, false)) { ... this branch is UNlikely ... } if (condition) [[likely]] { ... this branch is likely... } ``` Note that while clang provides `__builtin_expect`, it does not convert it to `llvm.expect.i8`. Instead, it looks at the surrounding code and if there is a condition, emits branch weight metadata for conditional branches. ### Design Implementing `likely`/`unlikely` type of functions properly to emit branch weights would add significant complexity to the compiler. Additionally, these functions are not easy to use with `match` arms. Replacing the functions with attributes is easier to implement and will also work with `match`. A question remains whether these attributes should be named `likely`/`unlikely` as in C++, or if we could reuse the already existing `#[cold]` attribute. `#[cold]` has the same meaning as `unlikely`, i.e., marking the slow path, but it can currently only be used on entire functions. I personally prefer `#[cold]` because it already exists in Rust and is a short word that looks better in code. It has one disadvantage though. This code: ``` if cond #[likely] { ... } ``` becomes: ``` if cond { ... } #[cold] { ... empty cold branch ... } ``` In this PR, I implemented the possibility to add `#[cold]` attribute on match arms. Use is as follows: ``` match x { #[cold] true => { ... } // the true arm is UNlikely _ => { ... } // the false arm is likely } ``` ### Limitations The implementation only works on bool, or integers with single value arm and an otherwise arm. Extending it to other types and to `if` statements should not be too difficult.
Merged
bors
added a commit
to rust-lang-ci/rust
that referenced
this issue
Nov 17, 2024
Likely unlikely fix RFC 1131 ( rust-lang#26179 ) added likely/unlikely intrinsics, but they have been broken for a while: rust-lang#96276 , rust-lang#96275 , rust-lang#88767 . This PR tries to fix them. Changes: - added a new `cold_path()` intrinsic - `likely()` and `unlikely()` changed to regular functions implemented using `cold_path()`
github-actions bot
pushed a commit
to rust-lang/miri
that referenced
this issue
Nov 18, 2024
Likely unlikely fix RFC 1131 ( rust-lang/rust#26179 ) added likely/unlikely intrinsics, but they have been broken for a while: rust-lang/rust#96276 , rust-lang/rust#96275 , rust-lang/rust#88767 . This PR tries to fix them. Changes: - added a new `cold_path()` intrinsic - `likely()` and `unlikely()` changed to regular functions implemented using `cold_path()`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When the
std::intrinsics::unlikely/likelyfunction is used, the codegen doesn't emitllvm.expectannotation in release mode. Curiously enough, in debug mode it does emit it.A minimal example:
Playground link here. In debug mode, LLVM IR contains
llvm.expect. In release mode, it doesn't.This has been previously noted here (maybe it indeed started with LLVM 13?) and more recently here.
I can take a look into this, if you can point me into the right direction. There is a codegen test for
likely/unlikely, but it uses-C no-prepopulate-passes, which does not correspond to what release (-O) does.The text was updated successfully, but these errors were encountered: