Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

f128 from_bits / to_bits sometimes gets reversed on ppc #125102

Closed
tgross35 opened this issue May 14, 2024 · 19 comments
Closed

f128 from_bits / to_bits sometimes gets reversed on ppc #125102

tgross35 opened this issue May 14, 2024 · 19 comments
Labels
A-ABI Area: Concerning the application binary interface (ABI) A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-bug Category: This is a bug. F-f16_and_f128 `#![feature(f16)]`, `#![feature(f128)]` llvm-fixed-upstream Issue expected to be fixed by the next major LLVM upgrade, or backported fixes T-libs Relevant to the library team, which will review and decide on the PR/issue.

Comments

@tgross35
Copy link
Contributor

It seems like using transmute causes the upper and lower 64-bit halves to be loaded reversed. More at https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/f128.20system.20libraries.20noncompliant.20platforms/near/438489330

#![feature(f128)]

#[no_mangle]
#[inline(never)]
fn add_entry(a: f128, b: f128) -> f128 {
    a + b
}

fn main() {
    let a = f128::from_bits(0x0);
    let b = f128::from_bits(0x1);
    dbg!(a, b);
    let c = add_entry(a, b);
    dbg!(c);
}

Espected b to print as 0x00000000000000000000000000000001, but with rustc add_test.rs -o add_test.rust --target powerpc64-unknown-linux-gnu -Clinker=powerpc64-linux-gnu-gcc -Ctarget-cpu=pwr9 it instead prints 0x00000000000000010000000000000000.

@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label May 14, 2024
@tgross35
Copy link
Contributor Author

tgross35 commented May 14, 2024

@rustbot label +F-f16_and_f128 +A-ABI +T-libs +C-bug -needs-triage +A-llvm

@rustbot rustbot added A-ABI Area: Concerning the application binary interface (ABI) C-bug Category: This is a bug. F-f16_and_f128 `#![feature(f16)]`, `#![feature(f128)]` T-libs Relevant to the library team, which will review and decide on the PR/issue. and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels May 14, 2024
@tgross35 tgross35 changed the title Transmute for from_bits / to_bits may not work on ppc f128 transmute for from_bits / to_bits may not work on ppc May 14, 2024
@RalfJung
Copy link
Member

RalfJung commented May 14, 2024

If we define f128 to be the corresponding IEEE float format, then the transmute is by definition correct. That float format defines the representation in memory and I think it's endianess must work out exactly the same as u128? Cc @eddyb @jcranmer @Muon

So at first this looks like a platform bug to me. Certainly we'd have to change a lot more than just from_bits / to_bits if this was deemed correct behavior -- codegen and Miri also need to know how to convert between the raw bits an a floating-point value. Do LLVM optimizations do anything to handle this?

@RalfJung
Copy link
Member

RalfJung commented May 14, 2024

Are you sure this is not an ABI bug, a mismatch between caller and callee in how the arguments are interpreted? That seems more likely to me than an in-memory float format with mixed endianess.

Does it make any difference if you remove the from_bits call from the equation, and do the transmute inline via a union instead?

#![feature(f128)]

#[no_mangle]
#[inline(never)]
fn add_entry(a: f128, b: f128) -> f128 {
    a + b
}

union U {
    i: u128,
    f: f128,
}

fn main() { unsafe {
    let a = U { i: 0x0 };
    let b = U { i: 0x1 };
    dbg!(a.f, b.f);
    let c = add_entry(a.f, b.f);
    dbg!(c);
} }

@tgross35
Copy link
Contributor Author

tgross35 commented May 14, 2024

Yeah there is definitely something more going on here, setting opt-level=3 makes the problem go away

@tgross35 tgross35 changed the title f128 transmute for from_bits / to_bits may not work on ppc f128 from_bits / to_bits sometimes gets reversed on ppc May 14, 2024
@tgross35
Copy link
Contributor Author

tgross35 commented May 14, 2024

Slightly reduced

$ cat transmute.rs
#![feature(f128)]

fn main() {
    let a = f128::from_bits(0x1);
    dbg!(a);
}
$ rustc transmute.rs --target powerpc64-unknown-linux-gnu -C linker=powerpc64-linux-gnu-gcc -Ctarget-cpu=pwr9 -o transmute.rs.ppc64.pwr9
$ qemu-ppc64 -L /usr/powerpc64-linux-gnu/ transmute.rs.ppc64.pwr9                                                                                                                                                                    
[transmute.rs:5:5] a = 0x00000000000000010000000000000000

Any idea how to narrow further?

@tgross35
Copy link
Contributor Author

tgross35 commented May 14, 2024

Your suggestion does indeed fix the issue:

$ cat transmute_union.rs                                                                                                                                                                                                             24-05-14 08:15:29
#![feature(f128)]

union U {
    i: u128,
    f: f128,
}

fn main() {
    let a = U { i: 0x1 };
    dbg!(unsafe { a.f });
}
$ rustc transmute_union.rs --target powerpc64-unknown-linux-gnu -C linker=powerpc64-linux-gnu-gcc -Ctarget-cpu=pwr9 -o transmute_union.rs.ppc64.pwr9
$ qemu-ppc64 -L /usr/powerpc64-linux-gnu/ transmute_union.rs.ppc64.pwr9
[transmute_union.rs:10:5] unsafe { a.f } = 0x00000000000000000000000000000001

edit: but turn on -O and it’s borked again. Interesting.

@RalfJung
Copy link
Member

Any idea how to narrow further?

I'd define from_bits locally in this crate and add more debugging to see if it's passing arguments to that function where things break, or returning them from that function.

@Muon
Copy link

Muon commented May 14, 2024

If we define f128 to be the corresponding IEEE float format, then the transmute is by definition correct. That float format defines the representation in memory and I think it's endianess must work out exactly the same as u128?

IEEE 754 does not specify memory layout. It just gives interpretations for 128-bit integers, but nothing prevents hardware from having, say, little-endian integer arithmetic and big-endian floating-point arithmetic. That said, AFAICT, PowerPC is big-endian for everything. More relevantly, however, PowerPC does not have 128-bit floating-point support, so this is being emulated in software anyway.

@Muon
Copy link

Muon commented May 14, 2024

Correction: Power9 does in fact have native IEEE 754 binary128 support, as part of VSX, according to its architecture manual. And indeed it is big endian. Older PowerPC doesn't, but I did see that the compilation command line did request Power9.

@RalfJung
Copy link
Member

RalfJung commented May 14, 2024

0x00000000000000010000000000000000 would indicate some sort of weird mixed-endian format. So I can't believe that that's the actual intended format of anything, it looks more like a bug (possible more than one bug) in argument passing... and possible optimizer bugs as well, given that with -O even the union-based variant breaks.

In fact maybe it's all an optimizer bug; from_bits is from the stdlib after all, so built with optimizations.

@tgross35
Copy link
Contributor Author

tgross35 commented May 14, 2024

@ecnelises or @lu-zero any chance you have any ideas here? It is looking like this has to be a bug in LLVM, just don't know exactly what.

@Muon
Copy link

Muon commented May 15, 2024

I saw this message from Qiu Chaofan on Zulip, and thought I'd test whether the memory representation of the smallest positive f128 differs from the representation of 1 as a u128.

I got an ICE due to unimplemented stuff in const eval in Rust, so I had to resort to C (gcc, clang). However, there were no differences in the representation.

This isn't conclusive yet since the two groups of 4 bytes in the middle could still be swapped around, so I did another test with no repeats there (gcc, clang). Again, no differences.

It seems that both GCC and Clang assume that u128 is big endian, just like f128, even if the loads and stores in that case are conducted as two lots of 64 bits.

@RalfJung
Copy link
Member

even if the loads and stores in that case are conducted as two lots of 64 bits.

But that should be an implementation detail, right, and not observable? When the two halves are strung together to form f128, endianess needs to be taken into account properly.

I think the best starting point is figuring out why the optimizer breaks

#![feature(f128)]

union U {
    i: u128,
    f: f128,
}

fn main() {
    let a = U { i: 0x1 };
    dbg!(unsafe { a.f });
}

I see no way that's not an LLVM bug, so it might be worth reporting there (after some further minimization to remove the debug machinery from the equation).

@Muon
Copy link

Muon commented May 15, 2024

even if the loads and stores in that case are conducted as two lots of 64 bits.

But that should be an implementation detail, right, and not observable? When the two halves are strung together to form f128, endianess needs to be taken into account properly.

Yes, that's right. I was just checking whether u128 was stored with the lower 64 bits before the higher 64 bits. That would mess it up and look mixed endian. But that's not the case. That is, u128 is stored big endian, so it's definitely an LLVM bug.

@tgross35
Copy link
Contributor Author

tgross35 commented May 15, 2024

Something must be quite wonky on the LLVM side here, accidentally tried to compile the same thing for 32-bit ppc with the pwr9 target-cpu and it gives me a SIGILL. Not sure if pwr9 is even valid or not but an error message would have been nice :)

$ rustc transmute.rs --target powerpc-unknown-linux-gnu -C linker=powerpc-linux-gnu-gcc -Ctarget-cpu=pwr9 -o transmute.rs.ppc.pwr9
Illegal instruction

For reference:

rustc 1.80.0-nightly (9c9b56879 2024-05-05)
binary: rustc
commit-hash: 9c9b568792ef20d8459c745345dd3e79b7c7fa8c
commit-date: 2024-05-05
host: x86_64-unknown-linux-gnu
release: 1.80.0-nightly
LLVM version: 18.1.4

bt via gdb:

(gdb) file rustc
Reading symbols from rustc...
(gdb) run transmute.rs --target powerpc-unknown-linux-gnu -C linker=powerpc-linux-gnu-gcc -Ctarget-cpu=pwr9 -o transmute.rs.ppc.pwr9
Starting program: /home/tmgross/.cargo/bin/rustc transmute.rs --target powerpc-unknown-linux-gnu -C linker=powerpc-linux-gnu-gcc -Ctarget-cpu=pwr9 -o transmute.rs.ppc.pwr9

This GDB supports auto-downloading debuginfo from the following URLs:
  <https://debuginfod.ubuntu.com>
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
process 8177 is executing new program: /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffe95ff6c0 (LWP 8189)]
[New Thread 0x7fffe8dff6c0 (LWP 8190)]
[New Thread 0x7fffe36a66c0 (LWP 8191)]
[New Thread 0x7fffe2fff6c0 (LWP 8192)]
[New Thread 0x7fffe27ff6c0 (LWP 8193)]
[New Thread 0x7fffe1fff6c0 (LWP 8194)]
[Thread 0x7fffe27ff6c0 (LWP 8193) exited]

Thread 7 "opt cgu.0" received signal SIGILL, Illegal instruction.
[Switching to Thread 0x7fffe1fff6c0 (LWP 8194)]
0x00007fffeee2a599 in llvm::PPCTargetLowering::LowerOperation(llvm::SDValue, llvm::SelectionDAG&) const ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
(gdb) b
Breakpoint 1 at 0x7fffeee2a599
(gdb) bt
#0  0x00007fffeee2a599 in llvm::PPCTargetLowering::LowerOperation(llvm::SDValue, llvm::SelectionDAG&) const ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#1  0x00007ffff0227d99 in llvm::TargetLowering::LowerOperationWrapper(llvm::SDNode*, llvm::SmallVectorImpl<llvm::SDValue>&, llvm::SelectionDAG&) const ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#2  0x00007ffff0c68ac0 in llvm::DAGTypeLegalizer::ExpandIntegerOperand(llvm::SDNode*, unsigned int) [clone .cold] ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#3  0x00007ffff0389b36 in llvm::DAGTypeLegalizer::run() [clone .warm] () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#4  0x00007fffefb71b09 in llvm::SelectionDAGISel::CodeGenAndEmitDAG() () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#5  0x00007fffefbd92c2 in llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#6  0x00007fffefb167ea in llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#7  0x00007fffeede2b1e in (anonymous namespace)::PPCDAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#8  0x00007fffefb0ee14 in llvm::FPPassManager::runOnFunction(llvm::Function&) () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#9  0x00007fffefb0e285 in llvm::FPPassManager::runOnModule(llvm::Module&) () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#10 0x00007ffff0074ca0 in llvm::legacy::PassManagerImpl::run(llvm::Module&) () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/libLLVM.so.18.1-rust-1.80.0-nightly
#11 0x00007ffff6b58d10 in LLVMRustWriteOutputFile () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-c823c964cb628bbb.so
#12 0x00007ffff6b58928 in rustc_codegen_llvm::back::write::write_output_file () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-c823c964cb628bbb.so
#13 0x00007ffff6b565af in rustc_codegen_llvm::back::write::codegen () from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-c823c964cb628bbb.so
#14 0x00007ffff6b56285 in rustc_codegen_ssa::back::write::finish_intra_module_work::<rustc_codegen_llvm::LlvmCodegenBackend> ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-c823c964cb628bbb.so
#15 0x00007ffff6b59377 in std::sys_common::backtrace::__rust_begin_short_backtrace::<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()> ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-c823c964cb628bbb.so
#16 0x00007ffff6b58e5b in <<std::thread::Builder>::spawn_unchecked_<<rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::ExtraBackendMethods>::spawn_named_thread<rustc_codegen_ssa::back::write::spawn_work<rustc_codegen_llvm::LlvmCodegenBackend>::{closure#0}, ()>::{closure#0}, ()>::{closure#2} as core::ops::function::FnOnce<()>>::call_once::{shim:vtable#0} ()
   from /home/tmgross/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-c823c964cb628bbb.so
#17 0x00007ffff19997bb in alloc::boxed::{impl#48}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2022
#18 alloc::boxed::{impl#48}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2022
#19 std::sys::pal::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/pal/unix/thread.rs:108
#20 0x00007ffff1760b5a in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
#21 0x00007ffff17f15fc in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

Edit: reported this one at llvm/llvm-project#92233

@tgross35
Copy link
Contributor Author

tgross35 commented May 15, 2024

I at least was able to reduce the original issue enough to remove dbg! but it could probably be reduced further. Reported to LLVM at llvm/llvm-project#92246

@rustbot rustbot added the A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. label May 16, 2024
@tgross35
Copy link
Contributor Author

tgross35 commented Aug 8, 2024

llvm/llvm-project#95931 was merged, which should fix this.

@tgross35 tgross35 added the llvm-fixed-upstream Issue expected to be fixed by the next major LLVM upgrade, or backported fixes label Aug 8, 2024
@tgross35
Copy link
Contributor Author

We should get this with the next LLVM19 bump (after the ongoing rc3 bump) if llvm/llvm-project#105623 merges.

@beetrees
Copy link
Contributor

beetrees commented Nov 1, 2024

This was fixed by #130212.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ABI Area: Concerning the application binary interface (ABI) A-LLVM Area: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues. C-bug Category: This is a bug. F-f16_and_f128 `#![feature(f16)]`, `#![feature(f128)]` llvm-fixed-upstream Issue expected to be fixed by the next major LLVM upgrade, or backported fixes T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

5 participants