1.77 produced release binaries die w/ SIGKILL on macos 14.4

[devjgm]

Setup

$ cargo new demo
     Created binary (application) `demo` package
$ cd demo/

This command works with 1.76

$ cargo +1.76 run --release
   Compiling demo v0.1.0 (/Users/greg/scratch/demo)
    Finished release [optimized] target(s) in 0.63s
     Running `target/release/demo`
Hello, world!

This command fails with SIGKILL with 1.77

$ cargo +1.77 run --release
   Compiling demo v0.1.0 (/Users/greg/scratch/demo)
    Finished release [optimized] target(s) in 0.47s
     Running `target/release/demo`
Killed: 9
$ ./target/release/demo 
Killed: 9

I expected to see this happen: I expected to see "Hello, world!" printed.

Instead, this happened: The attempt to execute target/release/demo was killed by SIGKILL.

Note: This problem only seems to happen with --release profile builds. cargo +1.77 run --profile dev works fine.

Meta

rustc --version --verbose:

$ rustc --version --verbose
rustc 1.77.0 (aedd173a2 2024-03-17)
binary: rustc
commit-hash: aedd173a2c086e558c2b66d3743b344f977621a7
commit-date: 2024-03-17
host: aarch64-apple-darwin
release: 1.77.0
LLVM version: 17.0.6

$ sw_vers 
ProductName:            macOS
ProductVersion:         14.4
BuildVersion:           23E214

[devjgm]

The issue seems to have started in +nightly-2024-01-18:

$ cargo +nightly-2024-01-18 run --release 
    Finished release [optimized] target(s) in 0.16s
     Running `target/release/demo`
Killed: 9

But this one works using +nightly-2024-01-17

$ cargo +nightly-2024-01-17 run --release 
    Finished release [optimized] target(s) in 0.16s
     Running `target/release/demo`
Hello, world!

[joboet]

I have the exact same software versions, but I am unable to reproduce this. Could you try running the binaries under gdb/lldb to see where the SIGKILL gets generated?

[devjgm]

lldb reports that it's an invalid mach-o file.

$ cargo +1.77 run --release
    Finished release [optimized] target(s) in 0.02s
     Running `target/release/demo`
Killed: 9
$ lldb target/release/demo
(lldb) target create "target/release/demo"
Current executable set to '/Users/greg/scratch/demo/target/release/demo' (arm64).
(lldb) r
error: Malformed Mach-o file
(lldb) 

However, objdump seems to show that the mach-o file is fine:

$ file target/release/demo
target/release/demo: Mach-O executable arm64
$ objdump -f target/release/demo

target/release/demo:     file format mach-o-arm64
architecture: aarch64, flags 0x00000012:
EXEC_P, HAS_SYMS
start address 0x0000000000001ad0

---

lldb works with a --profile dev binary

$ cargo +1.77 run --profile dev
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/demo`
Hello, world!
$ lldb target/debug/demo
(lldb) target create "target/debug/demo"
Current executable set to '/Users/greg/scratch/demo/target/debug/demo' (arm64).
(lldb) r
Process 14925 launched: '/Users/greg/scratch/demo/target/debug/demo' (arm64)
Hello, world!
Process 14925 exited with status = 0 (0x00000000) 
(lldb) 

[joboet]

That looks like a linker issue. Do you have any linker overrides (e.g. in ~/.cargo/config.toml)?

[devjgm]

Nothing there. I rm -rf ~/.cargo ~/.rustup and start from scratch installing from rustup.rs.

$ rustup show
Default host: aarch64-apple-darwin
rustup home:  /Users/greg/.rustup

installed toolchains
--------------------

stable-aarch64-apple-darwin (default)
1.76-aarch64-apple-darwin
1.77-aarch64-apple-darwin

active toolchain
----------------

stable-aarch64-apple-darwin (default)
rustc 1.77.0 (aedd173a2 2024-03-17)

/Users/greg/.cargo
├── bin
│   ├── cargo
│   ├── cargo-clippy
│   ├── cargo-fmt
│   ├── cargo-miri
│   ├── clippy-driver
│   ├── rls
│   ├── rust-analyzer
│   ├── rust-gdb
│   ├── rust-gdbgui
│   ├── rust-lldb
│   ├── rustc
│   ├── rustdoc
│   ├── rustfmt
│   └── rustup
├── env
└── env.fish

2 directories, 16 files

Still see the same SIGKILL from the OP.

---

In case it matters, on a MacBook Apple M2 Pro

[joboet]

Ok, so it's the default linker. What does ld -v say for you?

[devjgm]

@(#)PROGRAM:ld PROJECT:ld-1053.12
BUILD 15:45:29 Feb  3 2024
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
will use ld-classic for: armv6 armv7 armv7s arm64_32 i386 armv6m armv7k armv7m armv7em
LTO support using: LLVM version 15.0.0 (static support for 29, runtime is 29)
TAPI support using: Apple TAPI version 15.0.0 (tapi-1500.3.2.2)

[ehuss]

I have a suspicion it might involve having the wrong strip in PATH, which can happen with homebrew. which strip should be /usr/bin/strip.

[devjgm]

Good suspicion! Indeed, a homebrew strip is found first. However, tweaking the path doesn't seem to fix the problem:

$ which strip
/opt/homebrew/opt/binutils/bin/strip
$ PATH=/usr/bin:$PATH which strip
/usr/bin/strip
$ PATH=/usr/bin:$PATH cargo +1.77 run --release
    Finished release [optimized] target(s) in 0.02s
     Running `target/release/demo`
Killed: 9

[ehuss]

You will need to run cargo clean before trying to run cargo run again.

[devjgm]

UPDATE: I forgot to do a clean before re-doing the build. Indeed changing the path for strip does fix the issue:

$ cargo clean
     Removed 17 files, 1.0MiB total
$ PATH=/usr/bin:$PATH cargo +1.77 run --release
   Compiling demo v0.1.0 (/Users/greg/scratch/demo)
    Finished release [optimized] target(s) in 0.78s
     Running `target/release/demo`
Hello, world!

Thank you!

[devjgm]

In case it's useful, here's a diff of the good (left) vs bad (right) binaries.

One thing I noticed is that the bad one (right) has a magic number of 0xfeedface, which I believe should be used for 32-bit executables. So maybe that's the problem, not sure.

---

Thank you both very much for helping debug this! Much appreciated!

[CrazyHer]

It might be related to this issue:
https://blogs.oracle.com/java/post/java-on-macos-14-4
This can explain why you received SIGKILL

[devjgm]

It might be related to this issue: https://blogs.oracle.com/java/post/java-on-macos-14-4 This can explain why you received SIGKILL

I think that's a different issue.

The issue that @joboet and @ehuss helped track down was due to using my machine using strip from homebrew. That strip from homebrew binutils does not work correctly on macOS, apparently. Though it seems to have worked until Rust 1.77 (likely until rust-lang/cargo#13257).

It may be nice if cargo warned if /usr/bin/strip is not used, but I'll let you folks decide if there's anything to be done about this. For now, my problem has been diagnosed and fixed on my end.

Thanks again for the help, folks!