Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TSAN segmentation fault in __tsan::TraceAddEvent #1505

Open
liangjs opened this issue Mar 28, 2022 · 3 comments
Open

TSAN segmentation fault in __tsan::TraceAddEvent #1505

liangjs opened this issue Mar 28, 2022 · 3 comments

Comments

@liangjs
Copy link

liangjs commented Mar 28, 2022

I am using g++ 11.2.0 with compilation flags -fsanitize=thread.
The program crashes with a segmentation fault.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff74cc6a0 in __tsan::TraceAddEvent (addr=0, typ=__tsan::EventTypeMop, fs=..., thr=0x7ffff6eaf7c0) at /usr/src/debug/gcc/libsanitizer/tsan/tsan_rtl.h:872
872     /usr/src/debug/gcc/libsanitizer/tsan/tsan_rtl.h: No such file or directory.
LEGEND: STACK | HEAP | CODE | DATA | RWX | RODATA
───────────────────────────────────────────────────────────────────────[ REGISTERS ]────────────────────────────────────────────────────────────────────────
 RAX  0x600000000000 ◂— 0x0
 RBX  0x7ffff6eaf7c0 ◂— 0x1
 RCX  0xe
 RDX  0x8
 RDI  0x7ffff6687808 ◂— 0x1
 RSI  0x7ffff7459ff4 (__cxa_guard_release+116) ◂— mov    rsi, rax
 R8   0x1
 R9   0x7ffff75ee0e0 (__tsan::ctx_placeholder+524320) —▸ 0x7ffff6685000 ◂— 0x0
 R10  0x7ffff7459ff4 (__cxa_guard_release+116) ◂— mov    rsi, rax
 R11  0x7ffff6687800 —▸ 0x7ffff777fcf0 ◂— 0x10000
 R12  0x7ffff74e3e4f (__sanitizer::GetAltStackSize()+95) ◂— mov    rax, qword ptr [rip + 0x29bea2]
 R13  0x568000000000 ◂— 0x0
 R14  0x4
 R15  0x550000000000
 RBP  0x7ffff6687800 —▸ 0x7ffff777fcf0 ◂— 0x10000
 RSP  0x7fffffffde90 —▸ 0x568000000000 ◂— 0x0
 RIP  0x7ffff74cc6a0 ◂— mov    qword ptr [rdx + rax], 0
─────────────────────────────────────────────────────────────────────────[ DISASM ]─────────────────────────────────────────────────────────────────────────
 ► 0x7ffff74cc6a0    mov    qword ptr [rdx + rax], 0
   0x7ffff74cc6a8    mov    eax, dword ptr [rbx + 0x1c]
   0x7ffff74cc6ab    test   eax, eax
   0x7ffff74cc6ad    jne    0x7ffff74cc6ed                <0x7ffff74cc6ed>

   0x7ffff74cc6af    mov    edx, dword ptr [rbx + 0x270]
   0x7ffff74cc6b5    lea    rdi, [rbx + 0x270]
   0x7ffff74cc6bc    movabs rax, 0x3ffffffffff
   0x7ffff74cc6c6    and    rax, qword ptr [rbx]
   0x7ffff74cc6c9    mov    qword ptr [rbx + rdx*8 + 0x298], rax
   0x7ffff74cc6d1    lea    rdx, [rbp + 0x78]
   0x7ffff74cc6d5    mov    qword ptr [rbx + 8], rax
─────────────────────────────────────────────────────────────────────────[ STACK ]──────────────────────────────────────────────────────────────────────────
00:0000│ rsp 0x7fffffffde90 —▸ 0x568000000000 ◂— 0x0
01:0008│     0x7fffffffde98 —▸ 0x7ffff777fcf0 ◂— 0x10000
02:0010│     0x7fffffffdea0 —▸ 0x7ffff6eaf7c0 ◂— 0x1
03:0018│     0x7fffffffdea8 —▸ 0x7ffff745a003 (__cxa_guard_release+131) ◂— mov    dword ptr [rbx], 1
04:0020│     0x7fffffffdeb0 —▸ 0x7ffff6eaf7c0 ◂— 0x1
05:0028│     0x7fffffffdeb8 —▸ 0x7ffff74e3e4f (__sanitizer::GetAltStackSize()+95) ◂— mov    rax, qword ptr [rip + 0x29bea2]
06:0030│     0x7fffffffdec0 ◂— 0x0
07:0038│     0x7fffffffdec8 ◂— 0xca035c49e83a9400
───────────────────────────────────────────────────────────────────────[ BACKTRACE ]────────────────────────────────────────────────────────────────────────
 ► f 0   0x7ffff74cc6a0
   f 1   0x7ffff74cc6a0
   f 2   0x7ffff745a003 __cxa_guard_release+131
   f 3   0x7ffff74e3e4f __sanitizer::GetAltStackSize()+95
   f 4   0x7ffff74e4515 __sanitizer::SetAlternateSignalStack()+85
   f 5   0x7ffff74e4515 __sanitizer::SetAlternateSignalStack()+85
   f 6   0x7ffff74e4715
   f 7   0x7ffff74c05bb __tsan::Initialize(__tsan::ThreadState*)+1243
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
pwndbg> bt
#0  0x00007ffff74cc6a0 in __tsan::TraceAddEvent (addr=0, typ=__tsan::EventTypeMop, fs=..., thr=0x7ffff6eaf7c0) at /usr/src/debug/gcc/libsanitizer/tsan/tsan_rtl.h:872
#1  __tsan::Release (thr=thr@entry=0x7ffff6eaf7c0, pc=<optimized out>, addr=addr@entry=140737345223920) at /usr/src/debug/gcc/libsanitizer/tsan/tsan_rtl_mutex.cpp:453
#2  0x00007ffff745a003 in __cxa_guard_release (g=0x7ffff777fcf0 <guard variable for __sanitizer::GetAltStackSize()::kAltStackSize>) at /usr/src/debug/gcc/libsanitizer/tsan/tsan_interceptors_posix.cpp:882
#3  0x00007ffff74e3e4f in __sanitizer::GetAltStackSize () at /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cpp:170
#4  0x00007ffff74e4515 in __sanitizer::SetAlternateSignalStack () at /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cpp:183
#5  __sanitizer::SetAlternateSignalStack () at /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cpp:174
#6  0x00007ffff74e4715 in __sanitizer::InstallDeadlySignalHandlers (handler=handler@entry=0x7ffff74bec70 <__tsan::TsanOnDeadlySignal(int, void*, void*)>) at
 /usr/src/debug/gcc/libsanitizer/sanitizer_common/sanitizer_posix_libcdep.cpp:217
#7  0x00007ffff74c05bb in __tsan::Initialize (thr=0x7ffff6eaf7c0) at /usr/src/debug/gcc/libsanitizer/tsan/tsan_rtl.cpp:395
#8  0x00007ffff7fcc01e in _dl_init () from /lib64/ld-linux-x86-64.so.2
#9  0x00007ffff7fe396a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
@liangjs
Copy link
Author

liangjs commented Mar 28, 2022

I find some related issues but they don't provide any solutions.
#772
#746

@liangjs liangjs changed the title TSAN segmentation fault TSAN segmentation fault in __tsan::TraceAddEvent Mar 28, 2022
@dvyukov
Copy link
Contributor

dvyukov commented Mar 28, 2022

Currently this code looks as follows. I am not sure why would it call __cxa_guard_acquire/release. This may be a mis-compiled sanitizer runtime. If so, this is better reported to gcc. However, I am not sure version 11 is still supported.
Sanitizers runtime is only supported for the current upstream version and the current tsan version does not have TraceAddEvent. If you can reproduce this with ToT clang, please update the issue with that info.
Thanks.

static uptr GetAltStackSize() {
  // Note: since GLIBC_2.31, SIGSTKSZ may be a function call, so this may be
  // more costly that you think. However GetAltStackSize is only call 2-3 times
  // per thread so don't cache the evaluation.
  return SIGSTKSZ * 4;
}

void SetAlternateSignalStack() {
  stack_t altstack, oldstack;
  CHECK_EQ(0, sigaltstack(nullptr, &oldstack));
  // If the alternate stack is already in place, do nothing.
  // Android always sets an alternate stack, but it's too small for us.
  if (!SANITIZER_ANDROID && !(oldstack.ss_flags & SS_DISABLE)) return;
  // TODO(glider): the mapped stack should have the MAP_STACK flag in the
  // future. It is not required by man 2 sigaltstack now (they're using
  // malloc()).
  altstack.ss_size = GetAltStackSize();
  altstack.ss_sp = (char *)MmapOrDie(altstack.ss_size, __func__);
  altstack.ss_flags = 0;
  CHECK_EQ(0, sigaltstack(&altstack, nullptr));
}

@liangjs
Copy link
Author

liangjs commented Mar 28, 2022

The problem disappears with clang 14.

But when I test it with clang, I encounter another issue similar to #1477....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants