SIGSEGV when building p384 crate

[coder0xff]

I encountered this while working to make WebRTC build on Xtensa. WebRTC depends on a crate called p384. I'll try to collect more information, but figured I'd raise awareness before I go build eps-rs from sources.

Reproduction

1. Create a new project with cargo generate --vcs none esp-rs/esp-idf-template cargo and select the esp32s3 platform.
2. Add p384 = "0.13" to the dependencies in the generated Cargo.toml
3. Run cargo build

Meta

$ ~/.rustup/toolchains/esp/bin/rustc --version --verbose
rustc 1.76.0-nightly (88269fa9e 2024-02-09) (1.76.0.1)
binary: rustc
commit-hash: 88269fa9ed1d862991d52315f6d76d064407a5c0
commit-date: 2024-02-09
host: x86_64-unknown-linux-gnu
release: 1.76.0-nightly
LLVM version: 17.0.1

Error output

$ cargo build
    Updating crates.io index
   Compiling compiler_builtins v0.1.103
   ... truncated ...
   Compiling ecdsa v0.16.9
   Compiling p384 v0.13.0
error: rustc interrupted by SIGSEGV, printing backtrace

/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0xb31ef3)[0x7fed25f6fef3]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fed250f2520]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3141124)[0x7fed2857f124]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x1e88607)[0x7fed272c6607]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fc95bf)[0x7fed294075bf]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fcdcdf)[0x7fed2940bcdf]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fce286)[0x7fed2940c286]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd0456)[0x7fed2940e456]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd3c2b)[0x7fed29411c2b]

### cycle encountered after 9 frames with period 10
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
### recursed 24 times

/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]
/home/coder0xff/.rustup/toolchains/esp/lib/librustc_driver-3f9d3f3e4b606d93.so(+0x3fd1efe)[0x7fed2940fefe]

note: rustc unexpectedly overflowed its stack! this is a bug
note: maximum backtrace depth reached, frames may have been lost
note: we would appreciate a report at https://github.com/rust-lang/rust
note: backtrace dumped due to SIGSEGV! resuming signal
error: could not compile `p384` (lib)

Caused by:
  process didn't exit successfully: `/home/coder0xff/.rustup/toolchains/esp/bin/rustc --crate-name p384 --edition=2021 /home/coder0xff/.cargo/registry/src/index.crates.io-6f17d22bba15001f/p384-0.13.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=204 --crate-type lib --emit=dep-info,metadata,link -C opt-level=z -C embed-bitcode=no -C debuginfo=2 -C debug-assertions=on --cfg 'feature="alloc"' --cfg 'feature="arithmetic"' --cfg 'feature="default"' --cfg 'feature="digest"' --cfg 'feature="ecdh"' --cfg 'feature="ecdsa"' --cfg 'feature="ecdsa-core"' --cfg 'feature="pem"' --cfg 'feature="pkcs8"' --cfg 'feature="sha2"' --cfg 'feature="sha384"' --cfg 'feature="std"' -C metadata=2e68a4bd970b572b -C extra-filename=-2e68a4bd970b572b --out-dir /home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps --target xtensa-esp32s3-espidf -C linker=ldproxy -L dependency=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps -L dependency=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/debug/deps --extern 'noprelude:alloc=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/liballoc-b22e5f4e8b01c2d6.rmeta' --extern 'noprelude:compiler_builtins=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libcompiler_builtins-121cfdb51e9fde92.rmeta' --extern 'noprelude:core=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libcore-1aa44d494566ff80.rmeta' --extern ecdsa_core=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libecdsa-0491ecaf72d34706.rmeta --extern elliptic_curve=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libelliptic_curve-4375ae2636334ac2.rmeta --extern 'noprelude:panic_abort=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libpanic_abort-0d0b61faa520b01c.rmeta' --extern 'noprelude:panic_unwind=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libpanic_unwind-70901e9a2d4c2202.rmeta' --extern primeorder=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libprimeorder-2d62deab7b09cd8d.rmeta --extern 'noprelude:proc_macro=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libproc_macro-68ed28ed6d9bf590.rmeta' --extern sha2=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libsha2-6c9586251b0adcb4.rmeta --extern 'noprelude:std=/home/coder0xff/Dropbox/Documents/Projects/Community/src/community/esp32s3-p384-sandbox/target/xtensa-esp32s3-espidf/debug/deps/libstd-aa6a4fe89f59dded.rmeta' -Z unstable-options --cap-lints allow --cfg espidf_time64` (signal: 11, SIGSEGV: invalid memory reference)
warning: build failed, waiting for other jobs to finish...

[coder0xff]

I cut down the p384 crate until I had a minimal reproduction. Some observations:

• The SIGSEGV is happening in LLVM
• Initially, the Xtensa toolchains appeared to be the problem because the crash could not be duplicated by building p384 using x86_64 or RISC-V target toolchains. However, after identifying the minimal reproduction, and then making the minimal reproduction very large (that will make sense shortly), crashing can be reproduced even on the x86_64 target toolchain.
• 3000 variables (give or take) is enough to trip up xtensa-esp32s3-espidf.
• 10000 variables will kill any Rust toolchain
• The original code that triggers the bug is here: https://github.com/RustCrypto/elliptic-curves/blob/master/p384/src/arithmetic/scalar/p384_scalar_32.rs#L147
• Removing the unused variables in that code (the let mut variables are all dead code) shortens the function enough to compile on xtensa-esp32s3-espidf.

Since this is not an esp-rs/rust bug, I'll close this ticket and open a bug report in upstream Rust. I'll also submit a patch to p384 to clean up the dead code.

Directory Structure

.
├── .cargo
│   └── config.toml
├── Cargo.toml
├── rust-toolchain.toml
└── src
    └── lib.rs

2 directories, 4 files

Contents of: .cargo/config.toml

[build]
target = "xtensa-esp32s3-espidf"

[target.xtensa-esp32s3-espidf]
linker = "ldproxy"

[unstable]
build-std = ["std"]

Contents of: Cargo.toml

[package]
name = "p384"
version = "0.14.0-pre"
edition = "2021"

Contents of: rust-toolchain.toml

[toolchain]
channel = "esp"

Contents of: src/lib.rs

• This file has been truncated for readability
• The full file is at: https://pastebin.com/s72e8Akk

pub const fn break_me() {
    let _x1 = 0;
    let _x2 = 0;
    let _x3 = 0;
    let _x4 = 0;
    let _x5 = 0;
    let _x6 = 0;
    let _x7 = 0;
    let _x8 = 0;
    let _x9 = 0;
    let _x10 = 0;
.
.
.
    let _x9991 = 0;
    let _x9992 = 0;
    let _x9993 = 0;
    let _x9994 = 0;
    let _x9995 = 0;
    let _x9996 = 0;
    let _x9997 = 0;
    let _x9998 = 0;
    let _x9999 = 0;
}
 
pub const BREAK_ME: () = break_me();