Segmentation fault inside TailRecursionElimination.cpp(LLVM) while compiling libsyntax at stage1

[evilactually]

When building on Windows I'm getting a "Segmentation fault" in release mode, it seems to be happening inside tail call optimization inside LLVM. I found a workaround for now, I disabled this optimization and build completed without errors.

When I compile with LLVM debug information (with --disable-optimize-llvm) I get a different error, something to do with invalid instruction. Also enabling debug information on rustc itself (with --enable-debug) makes the problem go away. So it is difficult to reproduce with debugging information on. Nevertheless I'm just going to report the problem here.

SETUP

Windows 10 32-bit
Using MSYS2(with MinGW64 tools in PATH)
GCC 5.3.0
Commit hash: 303892e

BUILDING

./configure && make # (release build/single-threaded)

OUTPUT

...
rustc: i686-pc-windows-gnu/stage1/lib/rustlib/i686-pc-windows-gnu/lib/libsyntax
/c/Src/rust-rls/mk/target.mk:186: recipe for target 'i686-pc-windows-gnu/stage1/lib/rustlib/i686-pc-windows-gnu/lib/stamp.syntax' failed
make: *** [i686-pc-windows-gnu/stage1/lib/rustlib/i686-pc-windows-gnu/lib/stamp.syntax] Segmentation fault

FAILING COMMAND (getting it with VERBOSE=1 make"):

CFG_LLVM_LINKAGE_FILE=/c/Src/rust-rls/i686-pc-windows-gnu/rt/llvmdeps.rs PATH=/c/Src/rust-rls/i686-pc-windows-gnu/stage1/bin:/c/Src/rust-rls/i686-pc-windows-gnu/llvm/Release/lib:$PATH i686-pc-windows-gnu/stage1/bin/rustc.exe --cfg stage1 -O --cfg rtopt -C rpath -C prefer-dynamic --target=i686-pc-windows-gnu -L "i686-pc-windows-gnu/rt" -L native="C:\Src\rust-rls\i686-pc-windows-gnu\llvm\Release/lib" --out-dir i686-pc-windows-gnu/stage1/lib/rustlib/i686-pc-windows-gnu/lib -C extra-filename=-db5a760f src/libsyntax/lib.rs

FAILING FUNCTION

_ZN12_GLOBAL__N_112TailCallElim13runOnFunctionERN4llvm8FunctionE:   
  ...
   0x6f61ddc5 <+981>:   jne    0x6f61dd90 <_ZN12_GLOBAL__N_112TailCallElim13runOnFunctionERN4llvm8FunctionE+928>
   0x6f61ddc7 <+983>:   movb   $0x0,0x3c(%esp)
   0x6f61ddcc <+988>:   jmp    0x6f61ddf6 <_ZN12_GLOBAL__N_112TailCallElim13runOnFunctionERN4llvm8FunctionE+1030>
   0x6f61ddce <+990>:   xchg   %ax,%ax
   0x6f61ddd0 <+992>:   mov    -0x4(%eax),%edx
   0x6f61ddd3 <+995>:   mov    0x28(%esp),%ecx
   0x6f61ddd7 <+999>:   sub    $0x4,%eax
   0x6f61ddda <+1002>:  mov    %eax,0x78(%esp)
=> 0x6f61ddde <+1006>:  mov    0x20(%edx),%ebx  ; EDX was 0 so reading (EDX + 0x20) is an error
   0x6f61dde1 <+1009>:  mov    %ebp,0x4(%esp)
   0x6f61dde5 <+1013>:  mov    %edx,(%esp)
   ...

I could provide a memory dump, but I don't know how helpful it would be, as it would be too complex.

DETAILS

Symbol name _ZN12_GLOBAL__N_112TailCallElim13runOnFunctionERN4llvm8FunctionE corresponds to method TailCallElim::runOnFunction in TailRecursionElimination.cpp file. In fact the crash happens inside another function TailCallElim::markTails, which is called from runOnFunction:

 do {
    for (auto &I : *BB) {
      if (Tracker.EscapePoints.count(&I))
        Escaped = ESCAPED;
=>    CallInst *CI = dyn_cast<CallInst>(&I);     // Crash happens here somewhere
=>    if (!CI || CI->isTailCall())           // It's hard to tell since assembly was optimized
        continue;

      bool IsNoTail = CI->isNoTailCall();
      if (!IsNoTail && CI->doesNotAccessMemory()) {

This happens while compiling libsyntax. By calling "getName" on "Function", I could find out what function was being processed

_ZN51Box$LT$$u5b$ptr..P$LT$ast..TypeBinding$GT$$u5d$$GT$10drop.5622017hb9abb3e15

Looks like it was some internal function for TypeBinding struct in ast.rs

I don't have any more information at this time, but I may be able to collect more later.

WORKAROUND

Add return false; to top of TailCallElim::runOnFunction. This will effectively skip the optimization

[Aatch]

Could you try doing whatever it was you did to reproduce, but enable LLVM assertions? Normally segmentation faults are prevented by the assertions (which cause the program to crash anyway), but if there isn't a guarding one for this, it may be a bug in LLVM.

[alexcrichton]

This looks suspiciously similar to #28260 which is unfortunately just caused by using newer gcc binaries to compile LLVM. Right now we currently require 4.9 ish to compile successfully (it's what the bots are using)

[evilactually]

Huh, yes it does sound like my problem, but like I said I did manage to build, just without tail call optimizations. I will try with 4.9 later. I'm still waiting for build with assertions to finish...

UPDATE:
Rebuilding with assertions produced no effect.

It looks like a very good bug report was already sent to LLVM, with a minimal case and everything, and it's still not fixed...

[evilactually]

It built correctly using 4.9.3. In my defense, the reason why I was using newer GCC was because 011a23e removed the mention of 4.9 from README. You can close this, it's definitely the same problem as old #28260. Also the "workaround" was not sufficient, the rusc that I've built with gcc 5.3.0 was failing tests and working very unreliably.

[alexcrichton]

Ok, thanks for the verification @vlsh! I'll close this in favor of that.