Rust "forgets" intel attribute on assembly code sometimes

[dbartussek]

I tried this code:

#![no_std]
#![no_main]
#![feature(asm)]
#![feature(lang_items)]

///! A small reproduction case
///!
///! Build with `cargo build`
///!
///! Notes:
///! - setting opt-level to 0 or 1 compiles
///! - opt-level 2, 3, "s", "z" cause the error
///! - a release build works just fine, just a debug build with optimization has the issue

use core::mem::{ManuallyDrop, MaybeUninit};
use core::panic::PanicInfo;

pub unsafe fn call_with_stack<T>(
    arg: &mut T,
    function: extern "sysv64" fn(&mut T) -> (),
    stack: *mut u8,
) {
    asm!(r#"
    mov rbp, rsp
    mov rsp, $2

    call $1

    mov rsp, rbp
    "#
    : // Return values
    : "{rdi}"(arg), "r"(function), "r"(stack) // Arguments
    : "rbp", "cc", "memory" // Clobbers
    : "volatile", "intel" // Options
    );
}

/// Calls a closure and returns the result
///
/// This function is unsafe because it changes the stack pointer to stack.
/// stack must be suitable to be used as a stack pointer on the target system.
pub unsafe fn call_closure_with_stack<F, R>(closure: F, stack: *mut u8) -> R
where
    F: FnOnce() -> R,
{
    extern "sysv64" fn inner<F, R>(data: &mut (ManuallyDrop<F>, MaybeUninit<R>))
    where
        F: FnOnce() -> R,
    {
        let result = {
            // Read the closure from context, taking ownership of it
            let function = unsafe { ManuallyDrop::take(&mut data.0) };

            // Call the closure.
            // This consumes it and returns the result
            function()
        };

        // Write the result into the context
        data.1 = MaybeUninit::new(result);
    }

    // The context contains the closure and uninitialized memory for the return value
    let mut context = (ManuallyDrop::new(closure), MaybeUninit::uninit());

    call_with_stack(
        &mut context,
        // We create a new, internal function that does not close over anything
        // and takes a context reference as its argument
        inner,
        stack,
    );

    // Read the result from the context
    // No values are in the context anymore afterwards
    context.1.assume_init()
}

#[no_mangle]
pub unsafe fn _start(stack: *mut u8) {
    call_closure_with_stack(|| (), stack);
}

#[no_mangle]
pub unsafe fn efi_main(stack: *mut u8) {
    call_closure_with_stack(|| (), stack);
}

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

#[lang = "eh_personality"] extern fn eh_personality() {}

I expected to see this happen:
The inline assembly should be compiled in intel syntax

Instead, this happened:
Under some circumstances, Rust seems to forget about the intel attribute and tries to compile it as AT&T assembly.
I have not found a consistent reason for this, if the code compiles or not changes seemingly at random with different compile options or code reordering.
Removing call_closure_with_stack also made resulted in no error, but I am not sure if it is the cause or just massages the code in such a way that the issue goes silent.

full cargo error message with --verbose

   Compiling asm_test v0.1.0 (C:\Users\Dario\RustProjects\asm_test)
     Running `rustc --crate-name asm_test --edition=2018 'src\main.rs' --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=s -C debuginfo=2 -C debug-assertions=on -C metadata=ff8ce8b6214d71b9 --out-dir 'C:\Users\Dario\RustProjects\asm_test\target\debug\deps' -C 'incremental=C:\Users\Dario\RustProjects\asm_test\target\debug\incremental' -L 'dependency=C:\Users\Dario\RustProjects\asm_test\target\debug\deps' -Ctarget-feature=+crt-static`
error: <inline asm>:2:5: error: unknown use of instruction mnemonic without a size suffix
    mov rbp, rsp
    ^

  --> src\main.rs:23:5
   |
23 | /     asm!(r#"
24 | |     mov rbp, rsp
25 | |     mov rsp, $2
26 | |
...  |
34 | |     : "volatile", "intel" // Options
35 | |     );
   | |______^

error: <inline asm>:5:5: error: unknown use of instruction mnemonic without a size suffix
    call %rax
    ^

  --> src\main.rs:23:5
   |
23 | /     asm!(r#"
24 | |     mov rbp, rsp
25 | |     mov rsp, $2
26 | |
...  |
34 | |     : "volatile", "intel" // Options
35 | |     );
   | |______^

error: <inline asm>:7:5: error: unknown use of instruction mnemonic without a size suffix
    mov rsp, rbp
    ^

  --> src\main.rs:23:5
   |
23 | /     asm!(r#"
24 | |     mov rbp, rsp
25 | |     mov rsp, $2
26 | |
...  |
34 | |     : "volatile", "intel" // Options
35 | |     );
   | |______^

error: aborting due to 3 previous errors

error: could not compile `asm_test`.

Caused by:
  process didn't exit successfully: `rustc --crate-name asm_test --edition=2018 'src\main.rs' --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C opt-level=s -C debuginfo=2 -C debug-assertions=on -C metadata=ff8ce8b6214d71b9 --out-dir 'C:\Users\Dario\RustProjects\asm_test\target\debug\deps' -C 'incremental=C:\Users\Dario\RustProjects\asm_test\target\debug\incremental' -L 'dependency=C:\Users\Dario\RustProjects\asm_test\target\debug\deps' -Ctarget-feature=+crt-static` (exit code: 1)

Meta

rustc --version --verbose:
Windows:

rustc 1.44.0-nightly (f509b26a7 2020-03-18)
binary: rustc
commit-hash: f509b26a7730d721ef87423a72b3fdf8724b4afa
commit-date: 2020-03-18
host: x86_64-pc-windows-msvc
release: 1.44.0-nightly
LLVM version: 9.0

Linux:

rustc 1.44.0-nightly (f509b26a7 2020-03-18)
binary: rustc
commit-hash: f509b26a7730d721ef87423a72b3fdf8724b4afa
commit-date: 2020-03-18
host: x86_64-unknown-linux-gnu
release: 1.44.0-nightly
LLVM version: 9.0

[tmiasko]

ValueMapper used during ThinLTO import does not preserve assembly dialect when remapping types (the last argument of InlineAsm::get, which is not provided here, defaults to AD_ATT) https://github.com/rust-lang/llvm-project/blob/9f65ad057357b307180955831968f79e74090a90/llvm/lib/Transforms/Utils/ValueMapper.cpp#L371-L372

Using -Clto=off should avoid the issue.

[dbartussek]

Interesting. This should probably be more prominently documented. I can't find a single reference to this issue online. No resource about inline assembly told me I had to turn off lto to get it to work.

[bjorn3]

This is a bug in LLVM. Inline assembly should just work with LTO.

[dbartussek]

Is this a known issue in LLVM? I can't find any mention of this anywhere else

[bjorn3]

https://bugs.llvm.org/buglist.cgi?quicksearch=lto%20inline%20assembly

There are several bugs for inline assembly and LTO interaction, but this one doesn't seem to be reported yet. Somebody with an account for the LLVM bug tracker will have to do this.

[tmiasko]

https://bugs.llvm.org/show_bug.cgi?id=45291

[Amanieu]

This seems to have been lying in bugzilla for a while so I submitted the patch to LLVM's code review: https://reviews.llvm.org/D80066

[Amanieu]

This was fixed by rust-lang/llvm-project#59, and the submodule update got pulled in by #69171.