Tracking issue for link_llvm_intrinsics

[aturon]

Tracks stabilization for the link_llvm_intrinsics feature, used via #[link_name="llvm.*"].

Edit: As this is obviously back-end specific, it will most likely never be stabilized.

[eefriedman]

We should just kill this off; there aren't any in-tree users, and it would be impossible to stabilize because LLVM doesn't make any stability promises about intrinsics.

[gnzlbg]

I am using link_llvm_intrinsics to work around missing intrinsics in rustc.

Some of these intrinsics belong in rustc, but even those would either remain unstable for a long period of time, or never be stabilized. This feature allows using these intrinsics out of tree for the benefit of the users of a nightly compiler (performance-wise).

[cramertj]

@gnzlbg @jonhoo Are you still using this feature? Can it be removed?

[jonhoo]

I've since switched to std::instrinsics::prefetch_read, but I believe @aweinstock314's prefetch crate still depends on it.

[cramertj]

@aweinstock314 Are you still actively using this feature?

[gnzlbg]

@cramertj the stdsimd crate uses this feature for basically every single intrinsic. So, yes, we are using this feature, and using it more every day. Pinging: @alexcrichton .

The idea is to move the stdsimd crate into std and core, as a way to provide the functionality this feature gives to stable users. The prefetch crate can then be ported to use stdsimd and will work on stable. Maybe one could ping its author to see if they could already port to the stdsimd crate.

[alexcrichton]

@cramertj @gnzlbg afaik this feature cannot be removed because it's being used to implement the standard library (and also the stdsimd crate destined for libstd). I'd pesonally see this as a perma-unstable issue.

[gnzlbg]

Is there a way to make features usable only in std ?

[SimonSapin]

Some unstable features are named something_internals to indicate that they’re an implementation detail and not on the path to stabilization. But a determined Nightly user can still write #![feature(something_internals)] and use them.

[scottmcm]

FWIW, this was really convenient as a way to try out whether an LLVM intrinsic is even helpful before doing all the work to make and propose a rustc intrinsic for it.

[aweinstock314]

I'm using this in https://crates.io/crates/prefetch, as mentioned; I'm also using it (through https://crates.io/crates/llvmint) in https://github.com/aweinstock314/libgarble-rust for AES-NI (inline assembly was generating suboptimal assembly relative to intrinsics). I don't think my usage depends on them being eventually stabilised, so the _internals feature flag would satisfy my usage.

[gnzlbg]

The AES-NI intrinsics might become stabilized in the future via stdsimd (see rust-lang/stdarch#295). The stdsimd crate provides some prefetching intrinsics already, but maybe the LLVM prefetch intrinsic should be exposed in core::intrinsic.

[scottmcm]

As a concrete example of why I'd like this to stick around as forever-unstable, I could do the following to experiment with the new intrinsics mentioned in #55286 without a code change:

#![feature(link_llvm_intrinsics)]
extern {
    #[link_name="llvm.sadd.sat.i32"]
    fn add_sat_i32(x: i32, y: i32) -> i32;
    #[link_name="llvm.uadd.sat.i32"]
    fn add_sat_u32(x: u32, y: u32) -> u32;
}
pub unsafe fn test_sfold(x: i32) -> i32 {
    add_sat_i32(add_sat_i32(x, 10), 20)
}
pub unsafe fn test_ufold(x: u32) -> u32 {
    add_sat_u32(add_sat_u32(x, 10), 20)
}

[npmccallum]

Does anyone know how to represent the LLVM token type with this feature?

[gnzlbg]

This feature only lets you link a rust function to an LLVM intrinsic function, but it doesn't let you use types that aren't available in Rust. So if you can't represent it with a Rust type, you can't do so with this feature, and might need to either add a rustc intrinsic, or a rustc type. Adding intrinsics is usually easier.

[ghost]

If this feature can be removed, how will it be removed? I think rustc doesn't do anything special with llvm.* extern functions -- it just emits the feature-gate error if the feature is not active:

rust/compiler/rustc_ast_passes/src/feature_gate.rs

Lines 458 to 467 in 77b996e

	let links_to_llvm =
	link_name.map_or(false, |val| val.as_str().starts_with("llvm."));
	if links_to_llvm {
	gate_feature_post!(
	&self,
	link_llvm_intrinsics,
	i.span,
	"linking to LLVM intrinsics is experimental"
	);
	}

[bjorn3]

If this feature can be removed, how will it be removed?

It would turn into a hard error I would guess.

[ghost]

I just wanted to add that I have never been able to make it work the same as _ReturnAddress()

Rust, Not working, I should get the same address as I get with _ReturnAddress()...

extern "C" {
    #[link_name = "llvm.returnaddress"]
    fn return_address(level: i32) -> *const i8;
}

println!("return_addresss is: {:#X?}", return_address(0));

C++, Working fine, I get the correct return address of my function.

_ReturnAddress()

printf("_ReturnAddress is: 0x%p\n", _ReturnAddress());

[bjorn3]

What do you actually need it for? Also what would you expect it to return in case a part of a function is outlined by LLVM to improve cache locality? Do you expect the return address of the outlined function or the function out of which it was outlined? And what do you expect it to return in case the current function was inlined? And what about targets where the return address doesn't exist at all, like wasm?

[ghost]

What do you actually need it for? Also what would you expect it to return in case a part of a function is outlined by LLVM to improve cache locality? Do you expect the return address of the outlined function or the function out of which it was outlined? And what do you expect it to return in case the current function was inlined? And what about targets where the return address doesn't exist at all, like wasm?

Hey @bjorn3, thanks for your answer and asking to clarify a couple of points, well foremost, I am not expert with both Rust/LLVM, but I really enjoy Rust and the community behind it, and want it to be my main programming language and drop C/C++.

Nevertheless, I do know my work environment well which is mainly reverse-engineering and memory editing, thus creating software programs that affect live process.

So I will surely modify this answer later, but for the moment I will describe my exact use case, which is really simple:

1. I'm building my Rust library as cdylib.
2. I have many functions to be called on injection of my library, so I use DllMain entry point function (bare minimum).
3. I'm hooking a specific function using a detour library (here, detour-rs).

Now we are getting to the point where this is problematic in Rust:

4. Once I intercepted my function using a detour, I have to control the call of other functions inside it.
5. I'm doing pattern scanning to get the memory address of a specific function inside a module of my process, then back to my hooked function, I have to compare the return address of my hooked function with the memory address I got before with pattern scanning, if the return address is equal to it, I would like to return false or true depending on the context.

And what do you expect it to return in case the current function was inlined?

I make sure that my hooked function is preceded by #[inline(never)] just to be sure (with/without the attribute, it doesn't change anything at the moment, which means the compiler does not inline my function by default), once again this is for my personal use case, and I couldn't imagine a case where my hooked function had to be inlined (as this is counter-productive).

And what about targets where the return address doesn't exist at all, like wasm?

In any case, I am pretty confident that this feature should stay as forever-unstable, like for the fact you mentioned that return address doesn't even exist at all for some target. But what I would like, is to at least make it work on Rust Nightly to have the same behavior as it currently is on C/C++ (only tested the _ReturnAddress intrinsic with MSVC).

[bjorn3]

Thanks for the detailed explanation of your use case!

On nightly

#![feature(link_llvm_intrinsics)]

extern "C" {
    #[link_name = "llvm.returnaddress"]
    fn return_address(level: i32) -> *const i8;
}

fn main() {
    println!("return_address is:  {:#X?}", unsafe { return_address(0) });
    println!("address of main is: {:#X?}", main as fn());
}

works for me and prints

return_address is:  0x00005608f3f89c13
address of main is: 0x00005608f3f89a70

What is the exact issue you have?

[ghost]

What is the exact issue you have?

The issue is that for the same use case, I am getting two different return address when I should get the same on both C++ and Rust.

Same code, same hooking method, different result.

I think that for my use case there is something more complex going on with LLVM, is your exemple this is a plain standard function in Rust, so you are getting the return address correctly.

In my case, I am calling a detoured function.

I think I should try to compile C++ code with Clang/LLVM and see if this is different too.

So many question now..,

are llvm.returnaddress and _ReturnAddress intrinsics supposed to have the same behavior?
am I getting the same call frame between the MSVC compiler and the LLVM compiler?

[bjorn3]

I tried both llvm.returnaddress from rust and _ReturnAddress from C++ with MSVC, and they seem to have the exact same behavior: https://rust.godbolt.org/z/G3Y4YxYrn

[ghost]

@bjorn3 I just want to confirm (after a lot of testing) that everything works great :), the issue was the hooking method, I switched from Microsoft Detours to a custom minhook implementation, and it is now working as intended 😄.