Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SIGSEGV/SIGILL while building rustc itself in x86_64 apple CI jobs #134220

Open
jieyouxu opened this issue Dec 12, 2024 · 6 comments
Open

SIGSEGV/SIGILL while building rustc itself in x86_64 apple CI jobs #134220

jieyouxu opened this issue Dec 12, 2024 · 6 comments
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) C-bug Category: This is a bug. CI-spurious-x86_64-apple-SIGSEGV-SIGILL CI spurious failure: SIGSEGV/SIGILL while building rustc itself on x86_64-apple I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) O-x86_64 Target: x86-64 processors (like x86_64-*) S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@jieyouxu
Copy link
Member

jieyouxu commented Dec 12, 2024 •
edited
Loading

This does not seem like an isolated case, see label https://github.com/rust-lang/rust/labels/CI-spurious-cg_llvm-SIGSEGV

In the revert PR #134207, non-compiler (and AFAICT unrelated changes) ran into a SIGSEGV while compiling rustc_codegen_llvm on the dist-x86_64-apple job.

CI job: https://github.com/rust-lang-ci/rust/actions/runs/12298794250/job/34323085670
Log: #134207 (comment)

  [RUSTC-TIMING] rustc_codegen_llvm test:false 62.093
  rustc exited with signal: 11 (SIGSEGV)
  error: could not compile `rustc_codegen_llvm` (lib)
  
  Caused by:
    process didn't exit successfully: `/Users/runner/work/rust/rust/build/bootstrap/debug/rustc /Users/runner/work/rust/rust/build/bootstrap/debug/rustc --crate-name rustc_codegen_llvm --edition=2021 compiler/rustc_codegen_llvm/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C codegen-units=1 --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values())' -C metadata=31812fcdce1ed1a6 -C extra-filename=-31812fcdce1ed1a6 --out-dir /Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps --target x86_64-apple-darwin -L dependency=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps -L dependency=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/release/deps --extern bitflags=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libbitflags-a1ecb01bec05c607.rmeta --extern itertools=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libitertools-fc8482292e402829.rmeta --extern libc=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/liblibc-71418dd2bae71546.rmeta --extern measureme=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libmeasureme-65f26e237d9cbc69.rmeta --extern object=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libobject-b9fdd0c4eec82855.rmeta --extern rustc_demangle=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_demangle-59d4195c10f8ce59.rmeta --extern rustc_abi=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_abi-8c308e15a91667ae.rmeta --extern rustc_ast=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_ast-6307ddaffbb78c2c.rmeta --extern rustc_attr=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_attr-a64f29895039e85d.rmeta --extern rustc_codegen_ssa=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_codegen_ssa-471fc4cdea12e5aa.rmeta --extern rustc_data_structures=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_data_structures-ad6787aabac41fc7.rmeta --extern rustc_errors=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_errors-ae10afa2bd2798f4.rmeta --extern rustc_fluent_macro=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/release/deps/librustc_fluent_macro-e27f644eab9503e4.dylib --extern rustc_fs_util=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_fs_util-1af14c49c485068e.rmeta --extern rustc_hir=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_hir-ddd2bd35821eba08.rmeta --extern rustc_index=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_index-ba2f5cf42e80b6b3.rmeta --extern rustc_llvm=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_llvm-a806c45190fb0592.rmeta --extern rustc_macros=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/release/deps/librustc_macros-21f9064df7350bce.dylib --extern rustc_metadata=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_metadata-8c77a936e16e5034.rmeta --extern rustc_middle=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_middle-8709a3bcf14547b0.rmeta --extern rustc_query_system=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_query_system-531ae3841194b38a.rmeta --extern rustc_sanitizers=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_sanitizers-21a20f1a3f01e840.rmeta --extern rustc_session=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_session-63c14fae59e29aee.rmeta --extern rustc_span=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_span-2d8da5b611828dda.rmeta --extern rustc_symbol_mangling=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_symbol_mangling-595e7538c1ed6229.rmeta --extern rustc_target=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/librustc_target-bcb3e09908ca7102.rmeta --extern serde=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libserde-632f5b42d8037b7d.rmeta --extern serde_json=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libserde_json-b239b225d337f1f5.rmeta --extern smallvec=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libsmallvec-e8983336671b9a36.rmeta --extern tracing=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/deps/libtracing-6a46015c9b81a7be.rmeta --cfg=windows_raw_dylib -Csymbol-mangling-version=v0 -Zunstable-options '--check-cfg=cfg(bootstrap)' '--check-cfg=cfg(llvm_enzyme)' -Zmacro-backtrace -Csplit-debuginfo=unpacked '-Wrustc::internal' '-Drustc::symbol_intern_string_literal' -Wkeyword_idents_2024 -Wunsafe_op_in_unsafe_fn -Zosx-rpath-install-name '-Clink-args=-Wl,-rpath,@loader_path/../lib' -Zon-broken-pipe=kill -Zdylib-lto -Clto=thin -Cembed-bitcode=yes -Z binary-dep-depinfo -L native=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/build/psm-c80cca6b00966791/out -L native=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/build/blake3-dcfcf6f77ce37e35/out -L native=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/build/blake3-dcfcf6f77ce37e35/out -L native=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-rustc/x86_64-apple-darwin/release/build/rustc_llvm-d5c0fad7751ae19c/out -L native=/Users/runner/work/rust/rust/build/x86_64-apple-darwin/llvm/lib` (exit status: 254)
  warning: build failed, waiting for other jobs to finish...
  [RUSTC-TIMING] rustc_hir_analysis test:false 291.587

Unfortunately, I don't have a repro or details, and it might be spurious (which is still not good). Opening to record in case anyone else also run into this, and that this might be a genuine issue (like a miscompile or whatever).
@jieyouxu jieyouxu added the C-bug Category: This is a bug. label Dec 12, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Dec 12, 2024
@jieyouxu jieyouxu changed the title SIGSEGV while building cg_llvm SIGSEGV while building rustc_codegen_llvm in dist-x86_64-apple Dec 12, 2024
@jieyouxu jieyouxu added I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Dec 12, 2024
@jieyouxu
Copy link
Member Author

Tagging as A-spurious until proven otherwise.

@jieyouxu jieyouxu added the A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) label Dec 12, 2024
@jieyouxu jieyouxu mentioned this issue Dec 6, 2024
Open
@jieyouxu
Copy link
Member Author

jieyouxu commented Dec 12, 2024 •
edited
Loading

This is not an isolated instance, this has previously happened in #129984 (comment) on x86_64-apple-2.

@jieyouxu jieyouxu added the CI-spurious-x86_64-apple-SIGSEGV-SIGILL CI spurious failure: SIGSEGV/SIGILL while building rustc itself on x86_64-apple label Dec 12, 2024
@jieyouxu jieyouxu changed the title SIGSEGV while building rustc_codegen_llvm in dist-x86_64-apple SIGSEGV while building rustc_codegen_llvm in x86_64 apple CI jobs Dec 12, 2024
@jieyouxu jieyouxu added O-x86_64 Target: x86-64 processors (like x86_64-*) O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) labels Dec 12, 2024
@jieyouxu jieyouxu mentioned this issue Dec 13, 2024
Open
@jieyouxu
Copy link
Member Author

Possibly related but unclear if same root cause: #130060 (comment)

[RUSTC-TIMING] rustc_borrowck test:false 317.936
rustc exited with signal: 4 (SIGILL)

Caused by:
Caused by:
  process didn't exit successfully: [...] (exit status: 254)
[RUSTC-TIMING] rustc_hir_typeck test:false 331.010

@workingjubilee
Copy link
Member

cc @DianQK do you know of anything relevant?

@jieyouxu jieyouxu changed the title SIGSEGV while building rustc_codegen_llvm in x86_64 apple CI jobs SIGSEGV/SIGILL while building rustc_codegen_llvm in x86_64 apple CI jobs Dec 13, 2024
@jieyouxu jieyouxu changed the title SIGSEGV/SIGILL while building rustc_codegen_llvm in x86_64 apple CI jobs SIGSEGV/SIGILL while building rustc itself in x86_64 apple CI jobs Dec 13, 2024
@jieyouxu jieyouxu mentioned this issue Dec 13, 2024
Draft
bors added a commit to rust-lang-ci/rust that referenced this issue Dec 13, 2024
@bors
Auto merge of rust-lang#134275 - jieyouxu:exp-x86_64-apple-ci, r=<try>
6a8daf9 
[EXPERIMENTAL] Something's Wrong I Can Feel It (x86_64 apple funny business)

See rust-lang#134220

r? `@ghost`

try-job: dist-x86_64-apple
@DianQK
Copy link
Member

DianQK commented Dec 14, 2024

Do all errors occur after #133799? It's maybe rust-lang/llvm-project@104d0d1...1268e87#diff-e3b933e8c46156c0eaf7cbb67866b712f69a8484bc941d10acec9d4d17b9f061R15444.

@jieyouxu
Copy link
Member Author

jieyouxu commented Dec 14, 2024 •
edited
Loading

Do all errors occur after #133799? It's maybe rust-lang/llvm-project@104d0d1...1268e87#diff-e3b933e8c46156c0eaf7cbb67866b712f69a8484bc941d10acec9d4d17b9f061R15444.

No, there was a Sept instance, this would be too late for that (though it's not necessarily mutually exclusive)

bors added a commit to rust-lang-ci/rust that referenced this issue Dec 14, 2024
@bors
Auto merge of rust-lang#134275 - jieyouxu:exp-x86_64-apple-ci, r=<try>
61700e5 
[EXPERIMENTAL] Something's Wrong I Can Feel It (x86_64 apple funny business)

See rust-lang#134220

r? `@ghost`

try-job: dist-x86_64-apple-alt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) C-bug Category: This is a bug. CI-spurious-x86_64-apple-SIGSEGV-SIGILL CI spurious failure: SIGSEGV/SIGILL while building rustc itself on x86_64-apple I-crash Issue: The compiler crashes (SIGSEGV, SIGABRT, etc). Use I-ICE instead when the compiler panics. O-apple Operating system: Apple (macOS, iOS, tvOS, visionOS, watchOS) O-x86_64 Target: x86-64 processors (like x86_64-*) S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@DianQK @jieyouxu @workingjubilee @rustbot