Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCC 8.2.1 needs -fno-gnu-unique to work around std::make_shared defect due to libicuuc.so(DT_SYMBOLIC) #363

Closed
abyss7 opened this issue Apr 2, 2019 · 14 comments

Comments

@abyss7
Copy link

abyss7 commented Apr 2, 2019

Observed behavior

ccls crashes:

14:01:31 indexer1         pipeline.cc:344 I parse gh/ch/dbms/src/Formats/FormatFactory.cpp
14:01:31 preamble     sema_manager.cc:734 I create session for gh/ch/dbms/src/Formats/FormatFactory.cpp
LLVM ERROR: out of memory
LLVMSymbolizer: error reading file: No such file or directory
#0 0x00007fe75b21d09e llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/lib64/libLLVM-7.so+0x9b409e)
#1 0x00007fe75b21b554 llvm::sys::RunSignalHandlers() (/lib64/libLLVM-7.so+0x9b2554)
#2 0x00007fe75b21b6d5 (/lib64/libLLVM-7.so+0x9b26d5)
#3 0x00007fe75a523030 __restore_rt (/lib64/libpthread.so.0+0x13030)
#4 0x00007fe75a38053f __GI_raise /usr/src/debug/glibc-2.28-60-g4d7af7815a/signal/../sysdeps/unix/sysv/linux/internal-signals.h:84:10
#5 0x00007fe75a36a895 __GI_abort /usr/src/debug/glibc-2.28-60-g4d7af7815a/stdlib/abort.c:81:7
#6 0x00007fe75b19fe34 llvm::report_bad_alloc_error(char const*, bool) (/lib64/libLLVM-7.so+0x936e34)
#7 0x00007fe75b1c4bf1 llvm::SmallVectorBase::grow_pod(void*, unsigned long, unsigned long) (/lib64/libLLVM-7.so+0x95bbf1)
#8 0x00007fe75b1fc0bd llvm::raw_svector_ostream::write_impl(char const*, unsigned long) (/lib64/libLLVM-7.so+0x9930bd)
#9 0x00007fe75b1fcb97 llvm::raw_ostream::write(char const*, unsigned long) (/lib64/libLLVM-7.so+0x993b97)
#10 0x00007fe75b1e895c llvm::Twine::print(llvm::raw_ostream&) const (/lib64/libLLVM-7.so+0x97f95c)
#11 0x00007fe75b1e8b80 llvm::Twine::toVector(llvm::SmallVectorImpl<char>&) const (/lib64/libLLVM-7.so+0x97fb80)
#12 0x00007fe75ea2c54f clang::vfs::InMemoryFileSystem::setCurrentWorkingDirectory(llvm::Twine const&) (/lib64/libclangBasic.so.7+0x1e254f)
#13 0x00007fe75ea2a52e clang::vfs::OverlayFileSystem::pushOverlay(llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>) (/lib64/libclangBasic.so.7+0x1e052e)
#14 0x00007fe75fe2c00e (/lib64/libclangFrontend.so.7+0x12500e)
#15 0x00007fe75fe2c1fb clang::PrecompiledPreamble::setupPreambleStorage(clang::PrecompiledPreamble::PCHStorage const&, clang::PreprocessorOptions&, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>&) (/lib64/libclangFrontend.so.7+0x1251fb)
#16 0x00007fe75fe2c696 clang::PrecompiledPreamble::configurePreamble(clang::PreambleBounds, clang::CompilerInvocation&, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>&, llvm::MemoryBuffer*) const (/lib64/libclangFrontend.so.7+0x125696)
#17 0x0000000000483c54 (ccls+0x483c54)
#18 0x000000000047fed8 (ccls+0x47fed8)
#19 0x00007fe75a51858e start_thread /usr/src/debug/glibc-2.28-60-g4d7af7815a/nptl/pthread_create.c:487:7
#20 0x00007fe75a4456a3 __GI___clone /usr/src/debug/glibc-2.28-60-g4d7af7815a/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:97:0
ccls server connection was closed
[Error - 2:01:40 PM] Connection to server got closed. Server will not be restarted.

Expected behavior

ccls should reindex new contents.

Steps to reproduce

  1. git checkout new revision.
  2. Open any cpp changed file in vscode.
  3. Try to use index in any way: hover cursor, find references, etc.

System information

  • ccls version (git describe --tags --long): 0.20181225.9-35-g1976fec5
  • clang version: clang version 7.0.1 (tags/RELEASE_701/final)
  • OS: Fedora 29
  • Editor: vscode
  • Language client (and version): vscode-ccls 0.1.26
@MaskRay
Copy link
Owner

MaskRay commented Apr 5, 2019

Have you tried building ccls with clang 8? If your distribution doesn't provide it, consider using prebuilt archives on http://releases.llvm.org/

#17 0x0000000000483c54 (ccls+0x483c54)
#18 0x000000000047fed8 (ccls+0x47fed8)

A -DCMAKE_BUILD_TYPE=Debug build will be helpful

@cbsirb
Copy link

cbsirb commented Apr 8, 2019

Had an similar problem (crash whenever a file was indexed), which was fixed by using -DCMAKE_CXX_COMPILER=clang

@0xC0FFEE
Copy link

0xC0FFEE commented Apr 8, 2019

Had an similar problem (crash whenever a file was indexed), which was fixed by using -DCMAKE_CXX_COMPILER=clang

I had this bug too and discovered by accident that using clang to compile ccls fixes this. Tested with clang 8.0.0 (works) and gcc 8.2.1 (crashes).

Here is the backtrace of the core dump:

Core was generated by `ccls/build/ccls -log-file=/tmp/ccls.log -v=1'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00005598515613c7 in std::type_info::operator== (this=0x7fe8f20aa950 <std::_Sp_make_shared_tag::_S_ti()::__tag>, __arg=...) at /usr/include/c++/8.2.1/typeinfo:123
123                   || (__name[0] != '*' &&
[Current thread is 1 (Thread 0x7fe8ecaef700 (LWP 5994))]
(gdb) bt
#0  0x00005598515613c7 in std::type_info::operator== (this=0x7fe8f20aa950 <std::_Sp_make_shared_tag::_S_ti()::__tag>, __arg=...) at /usr/include/c++/8.2.1/typeinfo:123
#1  0x0000559851567235 in std::_Sp_counted_ptr_inplace<llvm::sys::fs::detail::DirIterState, std::allocator<llvm::sys::fs::detail::DirIterState>, (__gnu_cxx::_Lock_policy)2>::_M_get_deleter (this=0x7fe8d8014be0,  type_info node=...)
    at /usr/include/c++/8.2.1/bits/shared_ptr_base.h:573
#2  0x00007fe8f35fa77b in ?? () from /usr/lib/libLLVM-8.so
#3  0x00007fe8f751d3b1 in ?? () from /usr/lib/libclangDriver.so.8
#4  0x00007fe8f751e360 in ?? () from /usr/lib/libclangDriver.so.8
#5  0x00007fe8f7537015 in ?? () from /usr/lib/libclangDriver.so.8
#6  0x00007fe8f7468b7b in clang::driver::Driver::getToolChain(llvm::opt::ArgList const&, llvm::Triple const&) const () from /usr/lib/libclangDriver.so.8
#7  0x00007fe8f74746a4 in clang::driver::Driver::BuildCompilation(llvm::ArrayRef<char const*>) () from /usr/lib/libclangDriver.so.8
#8  0x00007fe8f85b97a3 in clang::createInvocationFromCommandLine(llvm::ArrayRef<char const*>, llvm::IntrusiveRefCntPtr<clang::DiagnosticsEngine>, llvm::IntrusiveRefCntPtr<llvm::vfs::FileSystem>) () from /usr/lib/libclangFrontend.so.8
#9  0x0000559851559871 in ccls::BuildCompilerInvocation (main="test_tokenizer.cpp", args=std::vector of length 8, capacity 14 = {...}, VFS=...) at ccls/src/clang_tu.cc:107
#10 0x000055985158c098 in ccls::idx::Index (manager=0x7ffefa709de0, wfiles=0x7ffefa709890, vfs=0x7ffefa7098f0, opt_wdir="build", main="test_tokenizer.cpp", 
    args=std::vector of length 7, capacity 7 = {...}, remapped=std::vector of length 0, capacity 0, no_linkage=false, ok=@0x7fe8ecaee460: true) at ccls/src/indexer.cc:1234
#11 0x00005598515ecc86 in ccls::pipeline::(anonymous namespace)::Indexer_Parse (completion=0x7ffefa709de0, wfiles=0x7ffefa709890, project=0x7ffefa709830, vfs=0x7ffefa7098f0, matcher=...)
    at ccls/src/pipeline.cc:360
#12 0x00005598515edf62 in ccls::pipeline::Indexer_Main (manager=0x7ffefa709de0, vfs=0x7ffefa7098f0, project=0x7ffefa709830, wfiles=0x7ffefa709890) at ccls/src/pipeline.cc:471
#13 0x00005598516eff20 in ccls::(anonymous namespace)::Indexer (arg_=0x559851def1f0) at ccls/src/messages/initialize.cc:263
#14 0x00007fe8f28fca9d in start_thread () from /usr/lib/libpthread.so.0
#15 0x00007fe8f282ab23 in clone () from /usr/lib/libc.so.6

Edit: maybe this should be moved to its own issue?

@yssource
Copy link

yssource commented Apr 9, 2019

the similar problem happens to me also.

  • archlinuxcn/ccls-git 20190329-2
  • emacs27
clang version 8.0.0 (tags/RELEASE_800/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1
Found candidate GCC installation: /usr/lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Selected GCC installation: /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/8.2.1
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64

ccls server wont start

@yssource
Copy link

yssource commented Apr 9, 2019

10:07:06 ccls           initialize.cc:271 I initialize in directory /tmp with uri file:///tmp/
10:07:06 ccls           initialize.cc:294 I initializationOptions: {"compilationDatabaseCommand":"","compilationDatabaseDirectory":"","cache":{"directory":"/home/jimmy/.emacs.d/.cache/lsp-ccls","format":"binary","hierarchicalPath":false,"retainInMemory":2},"capabilities":{"documentOnTypeFormattingProvider":{"firstTriggerCharacter":"}","moreTriggerCharacter":[]},"foldingRangeProvider":true,"workspace":{"workspaceFolders":{"supported":true,"changeNotifications":true}}},"clang":{"excludeArgs":[],"extraArgs":[],"pathMappings":[],"resourceDir":""},"client":{"hierarchicalDocumentSymbolSupport":true,"linkSupport":true,"snippetSupport":true},"codeLens":{"localVariables":true},"completion":{"caseSensitivity":2,"detailedLabel":true,"dropOldRequests":true,"duplicateOptional":true,"filterAndSort":true,"include":{"blacklist":[],"maxPathSize":30,"suffixWhitelist":[".h",".hpp",".hh",".inc"],"whitelist":[]},"maxNum":100},"diagnostics":{"blacklist":[],"onChange":1000,"onOpen":0,"onSave":0,"spellChecking":true,"whitelist":[]},"highlight":{"largeFileSize":2097152,"lsRanges":false,"blacklist":[],"whitelist":[]},"index":{"blacklist":[],"comments":2,"initialNoLinkage":false,"initialBlacklist":[],"initialWhitelist":[],"maxInitializerLines":5,"multiVersion":0,"multiVersionBlacklist":[],"multiVersionWhitelist":[],"name":{"suppressUnwrittenScope":false},"onChange":false,"parametersInDeclarations":true,"threads":0,"trackDependency":2,"whitelist":[]},"request":{"timeout":5000},"session":{"maxNum":10},"workspaceSymbol":{"caseSensitivity":1,"maxNum":1000,"sort":true},"xref":{"maxNum":2000}}
10:07:06 ccls           initialize.cc:323 I use -resource-dir=/usr/lib/clang/8.0.0
10:07:06 ccls           initialize.cc:356 I workspace folder: /tmp/
10:07:06 ccls           initialize.cc:381 I start 4 indexers
10:07:06 ccls           initialize.cc:389 I dispatch initial index requests
10:07:06 ccls             pipeline.cc:478 I loaded project. Refresh semantic highlight for all working file.
10:07:06 preamble     sema_manager.cc:733 I create session for /tmp/hello/hello.cpp
10:07:06 indexer3         pipeline.cc:344 I parse /tmp/hello/hello.cpp

Process ccls stderr finished

@yssource
Copy link

yssource commented Apr 9, 2019

It solves my problem.

So, I think archlinuxcn/ccls-git 20190329-2 is built with gcc (GCC) 8.2.1 20181127, instead of clang version 8.0.0 (tags/RELEASE_800/final under my personal environment.

@wbthomason
Copy link

@yssource I can confirm that this also fixed the issue for me on Arch; by adding -DCMAKE_CXX_COMPILER=clang++ to the normal AUR PKGBUILD I was able to get a build of ccls that does not crash. It seems like that should be added to the official PKGBUILD.

@yhamdoud, you may also be interested in this.

@madscientist
Copy link
Contributor

Interesting: I have ccls built with GCC 8.1.0 and I don't see this issue. This GCC is one I built myself so maybe there's some configuration of the Fedora version of GCC that is causing this problem?

@yhamdoud
Copy link

yhamdoud commented Apr 9, 2019

@yssource I can confirm that this also fixed the issue for me on Arch; by adding -DCMAKE_CXX_COMPILER=clang++ to the normal AUR PKGBUILD I was able to get a build of ccls that does not crash. It seems like that should be added to the official PKGBUILD.

@yhamdoud, you may also be interested in this.

Thanks, this also worked for me.

@rogorido
Copy link

rogorido commented Apr 9, 2019

* -DCMAKE_CXX_COMPILER=clang++

also for me on archlinux...

@abyss7
Copy link
Author

abyss7 commented Apr 10, 2019

Have you tried building ccls with clang 8? If your distribution doesn't provide it, consider using prebuilt archives on http://releases.llvm.org/

#17 0x0000000000483c54 (ccls+0x483c54)
#18 0x000000000047fed8 (ccls+0x47fed8)

A -DCMAKE_BUILD_TYPE=Debug build will be helpful

I built it with Clang 8 and can't reproduce the issue anymore. Can you clarify, how the compiler may help in this situation?

@MaskRay MaskRay changed the title ccls crashes when underlying content changes outside of editor ccls crashes due to GCC 8.2.1 ABI mismatch Apr 11, 2019
@MaskRay MaskRay changed the title ccls crashes due to GCC 8.2.1 ABI mismatch ccls crashes due to GCC 8.2.1 std::_Sp_counted_ptr_inplace<...>::_M_get_deleter(std::type_info const&) ABI mismatch Apr 11, 2019
@MaskRay
Copy link
Owner

MaskRay commented Apr 12, 2019

I'll jot down some notes here and keep updating this comment.

_ZZNSt19_Sp_make_shared_tag5_S_tiEvE5__tag: STB_GNU_UNIQUE or STB_WEAK

If you build ccls with extra/clang

% readelf -W --dyn-syms ~/abs/ccls-git/pkg.good/ccls-git/usr/bin/ccls | grep _Sp_make_shared_tag
   412: 00000000001214a0    16 OBJECT  WEAK   DEFAULT   15 _ZZNSt19_Sp_make_shared_tag5_S_tiEvE5__tag
   507: 0000000000121486    24 OBJECT  WEAK   DEFAULT   15 _ZTSSt19_Sp_make_shared_tag

If you build ccls with gcc 8.2.1

% readelf -W --dyn-syms ~/abs/ccls-git/pkg.bad/ccls-git/usr/bin/ccls | grep _Sp_make_shared_tag
   370: 0000000000140608    16 OBJECT  UNIQUE DEFAULT   13 _ZZNSt19_Sp_make_shared_tag5_S_tiEvE5__tag
   386: 0000000000140530    24 OBJECT  WEAK   DEFAULT   13 _ZTSSt19_Sp_make_shared_tag

For std::shared_ptr to work correctly, all references to the symbol _ZTSSt19_Sp_make_shared_tag must resolve to the main executable.

Now the issue is that for GCC 8.2.1 compiled ccls, the _ZTSSt19_Sp_make_shared_tag reference from libLLVM-8.so unexpectedly resolves to libiccuc.so.64. This is due to some nasty interaction between STB_GNU_VERSION and DT_SYMBOLIC used by libicuuc.so.64. libicuuc links in C++ object files and thus it really should not use -Wl,-Bsymbolic.

In glibc/elf/rtld.c, _dl_relocate_object relocates libicuuc.so.64 before libLLVM-8.so. When processing libicuuc.so.64, its scope contains just itself (DT_SYMBOLIC). Since the main executable is not in the scope, the binding from _ZTSSt19_Sp_make_shared_tag to libicuuc.so.64 is added to the _ns_unique_sym_table table.

  • libicuuc.so.64 libLLVM-8.so libclang*.so: _ZTSSt19_Sp_make_shared_tag binds to libicuuc.so.64
  • ccls: _ZTSSt19_Sp_make_shared_tag binds to ccls
% readelf --dyn-syms /usr/lib/libLLVM-8.so | grep _Sp_make_shared_tag
  3677: 0000000002e2d4e8    16 OBJECT  UNIQUE DEFAULT   12 _ZZNSt19_Sp_make_shared_tag5_S_tiEvE5__tag@@LLVM_8
 13290: 0000000002e2d290    24 OBJECT  WEAK   DEFAULT   12 _ZTSSt19_Sp_make_shared_tag@@LLVM_8

# libLLVM-8.so needs libxml2.so.2 needs libicuuc.so.64
% readelf --dyn-syms /usr/lib/libicuuc.so.64 | grep _Sp_make_shared_tag
   351: 0000000000140950    16 OBJECT  UNIQUE DEFAULT   11 _ZZNSt19_Sp_make_shared_tag5_S_tiEvE5__tag
  1508: 0000000000140890    24 OBJECT  WEAK   DEFAULT   11 _ZTSSt19_Sp_make_shared_tag

All of the following cmake commands fix the issue.

cmake -H. -Bbuild -DCMAKE_CXX_FLAGS=-fno-gnu-unique # for g++ 8.2.1. clang doesn't have this option
cmake -H. -Bbuild -DCMAKE_CXX_COMPILER=/usr/bin/clang++
cmake -H. -Bbuild -DCMAKE_EXE_LINKER_FLAGS='-fuse-ld=gold -Wl,--no-gnu-unique' # gcc 9 has -fuse-ld=lld

Or, locate the _ZZNSt19_Sp_make_shared_tag5_S_tiEvE5__tag entry in .dynsym, change its type from STB_GNU_UNIQUE to STB_GLOBAL or STB_WEAK.

libstdc++ shared_ptr 8.2.0 vs 8.2.1

libstdc++ shipped with GCC 8.2.1 includes this commit
gcc-mirror/gcc@d58f58b#diff-ebefa88e3ea88386df2fec339630cb2a

-#if __cpp_rtti
-   void* __p = _M_refcount._M_get_deleter(typeid(__tag));
-#else
-   void* __p = _M_refcount._M_get_deleter(_Sp_make_shared_tag::_S_ti());
-#endif
+void* __p = _M_refcount._M_get_deleter(_Sp_make_shared_tag::_S_ti());

We have __cpp_rtti (-frtti) builds. For GCC<8.2.1, the real type info object is passed and there is no issue. GCC 8.2.1 calls _Sp_make_shared_tag::_S_ti() and passes the fake type info object, __ti == typeif(_Sp_make_shared_tag) will segfault when the name string of __ti is dereferenced.

You cannot reproduce the issue with GCC 9. After gcc-mirror/gcc@3c1c2a8#diff-ebefa88e3ea88386df2fec339630cb2a , _M_get_deleter is no longer used by make_shared.

According to PR libstdc++/88782, when make_shared from GCC<8.2.1 and make_shared from GCC 8.2.1 are mixed, it can lead to segfault if the GCC<8.2.1 version is picked. Fortunately, this doesn't apply to releases.llvm.org llvm 8 prebuilt archives (GCC 8.2.1) because they are compiled with -fno-rtti.

  • GCC 8.1.0: -fno-rtti -fsanitize=alignment => fail
  • GCC < 8.2.1: clang -fvisibility-inlines-hidden (clang bug) => ODR violation
  • GCC < 8.2.1: -fno-rtti + -frtti => ODR violation
  • GCC 8.2.1: -frtti: make_shared from GCC<8.2.1 object file + make_shared from GCC 8.2.1 => ODR violation
  • GCC 8.2.1: libicuuc.so (DT_SYMBOLIC) + make_shared in .so + make_shared in exe => ODR violation

@MaskRay MaskRay changed the title ccls crashes due to GCC 8.2.1 std::_Sp_counted_ptr_inplace<...>::_M_get_deleter(std::type_info const&) ABI mismatch GCC 8.2.1 needs -fno-gnu-unique to work around std::make_shared defect Apr 13, 2019
@MaskRay MaskRay closed this as completed Apr 13, 2019
@MaskRay MaskRay changed the title GCC 8.2.1 needs -fno-gnu-unique to work around std::make_shared defect GCC 8.2.1 needs -fno-gnu-unique to work around std::make_shared defect due to libicuuc.so(DT_SYMBOLIC) Apr 13, 2019
@bastianbeischer
Copy link

Hm, if I understand the above correctly, then ccls built with GCC 8.3.0 should work fine, right? It includes the fix for https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88782. But I'm still seeing the same issues as the OP... Am I missing something?

@bastianbeischer
Copy link

Ah - now I understand: LLVM also has to be built with the new GCC version. Problem is fixed with Arch llvm package llvm-8.0.0-2. Thanks!

svenstaro pushed a commit to archlinux/svntogit-packages that referenced this issue Jul 21, 2020
Rebuild with GCC 8.3; fixes segfaults in clang's /usr/bin/modularize.

Some further explanation at the link bellow:

  MaskRay/ccls#363 (comment)

git-svn-id: file:///srv/repos/svn-packages/svn@352148 eb2447ed-0c53-47e4-bac8-5bc4a241df78
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants