Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spurious LLDB segfault (signal 11) #45230

Open
kennytm opened this issue Oct 12, 2017 · 2 comments
Open

Spurious LLDB segfault (signal 11) #45230

kennytm opened this issue Oct 12, 2017 · 2 comments
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.

Comments

@kennytm
Copy link
Member

kennytm commented Oct 12, 2017

Symptom: debuginfo-lldb tests failed with signal 11. Only happens on macOS (others use GDB) and usually i686, and only one test is failing.

Example logs (look for "status: signal: 11"):

[01:40:10] failures:
[01:40:10] 
[01:40:10] ---- [debuginfo-lldb] debuginfo/lexical-scopes-in-block-expression.rs stdout ----
[01:40:10] 	NOTE: compiletest thinks it is using LLDB version 360
[01:40:10] 
[01:40:10] error: Error while running LLDB
[01:40:10] status: signal: 11
[01:40:10] command: "/usr/bin/python" "/Users/travis/build/rust-lang/rust/src/etc/lldb_batchmode.py" "/Users/travis/build/rust-lang/rust/build/x86_64-apple-darwin/test/debuginfo/lexical-scopes-in-block-expression.stage2-x86_64-apple-darwin" "/Users/travis/build/rust-lang/rust/build/x86_64-apple-darwin/test/debuginfo/lexical-scopes-in-block-expression.debugger.script"
[01:40:10] stdout:
[01:40:10] ------------------------------------------
[01:40:10] LLDB batch-mode script
[01:40:10] ----------------------
...

Affected PRs:

Related: #42693 (GDB failed due to ERROR_NOT_ENOUGH_QUOTA on Windows)

@kennytm kennytm added A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC labels Oct 12, 2017
@Mark-Simulacrum
Copy link
Member

I wasn't able to reproduce the failure locally, but it seems like the problem might be with Python segfaulting, not LLDB. Below is the crashes log from #44967 (comment), and the first one seems to indicate that Python crashed. Maybe we can print that failure out or something? It seems possible that our Python batchmode script or our formatter script for LLDB has some thread race or something like that that causes the crash here....

�[0K$ ls $HOME/Library/Logs/DiagnosticReports/
Python_2017-10-12-005820_Traviss-Mac-28.crash
abort-link-to-unwinding-crates.stage2-x86_64-apple-darwin_2017-10-12-004319_Traviss-Mac-28.crash
abort.stage2-x86_64-apple-darwin_2017-10-12-004318_Traviss-Mac-28.crash
backtrace.stage2-x86_64-apple-darwin_2017-10-12-003318-1_Traviss-Mac-28.crash
backtrace.stage2-x86_64-apple-darwin_2017-10-12-003318_Traviss-Mac-28.crash
com.apple.CoreSimulator.CoreSimulatorService_2016-11-16-181716_Traviss-Mac.crash
issue-24313.stage2-x86_64-apple-darwin_2017-10-12-003832_Traviss-Mac-28.crash
lto-abort.stage2-x86_64-apple-darwin_2017-10-12-004323_Traviss-Mac-28.crash
out-of-stack.stage2-x86_64-apple-darwin_2017-10-12-004302-1_Traviss-Mac-28.crash
out-of-stack.stage2-x86_64-apple-darwin_2017-10-12-004302-2_Traviss-Mac-28.crash
out-of-stack.stage2-x86_64-apple-darwin_2017-10-12-004302_Traviss-Mac-28.crash
out-of-stack.stage2-x86_64-apple-darwin_2017-10-12-004303_Traviss-Mac-28.crash
running-with-no-runtime.stage2-x86_64-apple-darwin_2017-10-12-004425_Traviss-Mac-28.crash
segfault-no-out-of-stack.stage2-x86_64-apple-darwin_2017-10-12-004426_Traviss-Mac-28.crash
signal-exit-status.stage2-x86_64-apple-darwin_2017-10-12-004443_Traviss-Mac-28.crash
stack-probes-lto.stage2-x86_64-apple-darwin_2017-10-12-004515-1_Traviss-Mac-28.crash
stack-probes-lto.stage2-x86_64-apple-darwin_2017-10-12-004515_Traviss-Mac-28.crash
stack-probes.stage2-x86_64-apple-darwin_2017-10-12-004512-1_Traviss-Mac-28.crash
stack-probes.stage2-x86_64-apple-darwin_2017-10-12-004512_Traviss-Mac-28.crash

@kennytm
Copy link
Member Author

kennytm commented Nov 5, 2017

Ooooohhhhh that's an interesting point. I was puzzled that why there's no LLDB crash log, turns out it is Python crashing! I'll try to cat the crash log if the filename doesn't contain "stage2".

kennytm added a commit to kennytm/rust that referenced this issue Nov 5, 2017
kennytm added a commit to kennytm/rust that referenced this issue Nov 10, 2017
…crichton

Miscellaneous changes for CI, Docker and compiletest.

This PR contains 7 independent commits that improves interaction with CI, Docker and compiletest.

1. a4e5c91 — Forces a newline every 100 dots when testing in quiet mode. Prevents spurious timeouts when abusing the CI to test Android jobs.

2. 1b5aaf2 — Use vault.centos.org for dist-powerpc64le-linux, see rust-lang#45744.

3. 33400fb — Modify `src/ci/docker/run.sh` so that the docker images can be run from Docker Toolbox for Windows on Windows 7. I haven't checked the behavior of the newer Docker for Windows on Windows 10. Also, "can run" does not mean all the test can pass successfully (the UDP tests failed last time I checked)

4. d517668 — Don't emit a real warning the linker segfault, which affects UI tests like rust-lang#45489 (comment). Log it instead.

5. 51e2247 — During run-pass, trim the output if stdout/stderr exceeds 416 KB (top 160 KB + bottom 256 KB). This is an attempt to avoid spurious failures like rust-lang#45384 (comment)

6. 9cfdaba — Force `gem update --system` before deploy. This is an attempt to prevent spurious error rust-lang#44159.

7. eee10cc — Tries to print the crash log on macOS on failure. This is an attempt to debug rust-lang#45230.
@jonas-schievink jonas-schievink added T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. C-bug Category: This is a bug. and removed C-bug Category: This is a bug. labels Nov 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-spurious Area: Spurious failures in builds (spuriously == for no apparent reason) C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

3 participants