probe-stack=inline-asm generates wrong DWARF information

[YangKeao]

I'm the author of pprof-rs, which is a library to perf rust programs by sampling backtraces in signal handler. After #77885 merged, it causes Segmentation Fault. After investigating this bug, I found that it was caused by wrong DWARF information. In the old version of rustc, __rust_probstack doesn't have DWARF information, so when a signal hits __rust_probstack, _Unwind_Backtrace will get a short backtrace (only containing the signal handler parts) but not break. However, after using inline-asm with a wrong DWARF information, _Unwind_Backtrace will get a wrong context->ra and dereferencing it will cause a Segmentation Fault

I tried this code:

#[inline(never)]
fn big_stack() {
    let _two_page_stack = [2u8; 8192];
    return
}

I expected to see this happen: The dwarf information of this function works well

Instead, this happened: In the probe-stack part of codes, the dwarf information is wrong and _Unwind_Backtrace will cause Segmentation Fault by dereferencing an unexpected address (actually, the address is "0x0202020202020202").

Part of big_stack assembly codes:

349:00000000000f42f0 <_ZN9big_stack9big_stack17h672b13eba3b25c0dE>:
350-   f42f0:	48 81 ec 00 10 00 00 	sub    $0x1000,%rsp
351-   f42f7:	48 c7 04 24 00 00 00 	movq   $0x0,(%rsp)
352-   f42fe:	00 
353-   f42ff:	48 81 ec 00 10 00 00 	sub    $0x1000,%rsp
354-   f4306:	48 c7 04 24 00 00 00 	movq   $0x0,(%rsp)
355-   f430d:	00 
356-   f430e:	48 81 ec 08 00 00 00 	sub    $0x8,%rsp
357-   f4315:	48 8d 7c 24 08       	lea    0x8(%rsp),%rdi
358-   f431a:	be 01 00 00 00       	mov    $0x1,%esi
359-   f431f:	ba 00 20 00 00       	mov    $0x2000,%edx

The DWARF information about this part is quite simple:

00000138 00000014 0000010c FDE cie=00000030 pc=0004ceb0...0004cef1
  Format:       DWARF32
  DW_CFA_advance_loc: 37
  DW_CFA_def_cfa_offset: +8208
  DW_CFA_advance_loc: 27
  DW_CFA_def_cfa_offset: +8

Though, the probe-stack part of this codes modified the rsp, the corresponding DWARF information doesn't exist.

Should I also file an issue in LLVM bugzilla? It seems to be a problem of LLVM. (The simplest way to fix it in my imagination is to use another register to loop over the stack, but not using rsp directly.)

Meta

rustc --version --verbose:

rustc 1.51.0-nightly (1d0d76f8d 2021-01-24)
binary: rustc
commit-hash: 1d0d76f8dd4f5f6ecbeab575b87edaf1c9f56bb8
commit-date: 2021-01-24
host: x86_64-unknown-linux-gnu
release: 1.51.0-nightly
LLVM version: 11.0.1

[nagisa]

Please check with a more recent nightly version. I mildly recall hearing a similar issue which might have been fixed since.

[nagisa]

Upstream report: https://bugs.llvm.org/show_bug.cgi?id=49600

[YangKeao]

Please check with a more recent nightly version. I mildly recall hearing a similar issue which might have been fixed since.

Tried with rustc 1.52.0-nightly (107896c32 2021-03-15), and it's not fixed yet.

[YangKeao]

I've tried to fix this issue in llvm with this patch. After building rustc with this patch, it seems to be fine and the segmentation fault is also solved. I'll test it with bigger project (like tikv-server) tomorrow.

[apiraino]

Assigning P-critical as discussed as part of the Prioritization Working Group procedure and removing I-prioritize.

Will be mentioned during today's meeting.

@rustbot label -I-prioritize +P-critical

[pnkfelix]

discussed in T-compiler meeting.

We plan to revert PR #77885 (or at least a portion of it) on beta before next week's release, in order to buy us more time to evaluate tradeoffs here.

[apiraino]

Issue priority downgraded as discussed during the team compiler meeting

@rustbot label +P-high -P-critical

[pnkfelix]

discussed at team compiler meeting. A partial solution, addressing solely the case where the loop is unrolled, has landed in LLVM in https://reviews.llvm.org/D99579

We would probably be in favor of backporting D99579, even though it is only a partial solution to the problem here.

[pnkfelix]

Discussed in T-compiler meeting today.

Due to this issue, and issue #84667, we are reverting PR #77885 on master and beta.

I will look into making regression tests for these issues.

[pietroalbini]

A fix for this is available from 1.53 onwards (nightly and soon enough beta), and it's been backported to 1.52 stable. Closing the issue. If this is still happening please leave a comment and we'll reopen it!

[erikdesjardins]

Note: the underlying issue has been fixed upstream, see llvm/llvm-project#48944 (comment).

(Once LLVM 14 is released with those changes, I will revisit enabling inline asm stack probes on x86.)

Edit: actually #84667 / https://bugs.llvm.org/show_bug.cgi?id=50165 / llvm/llvm-project#49509 is still a problem

[bstrie]

@erikdesjardins You say that #84667 is still a problem, but that issue is closed. Should it be reopened?

[erikdesjardins]

I should've been clearer: #84667 occurred due to a bug in LLVM codegen related to inline-asm stack probes. It is not a problem for current-day Rust users, because we disabled inline-asm stack probes. But the underlying LLVM bug still exists, which prevents us from enabling inline-asm stack probes.

(I do not know if this means it should be reopened, presumably not, but hopefully this gives you enough to context to tell)