Indexing .. failed with errno=1 because in libclang/CXIndexDataConsumer.cpp, handleReferences may be passed nullptr DeclContext to

[MaskRay]

This is an issue of libclang 4.0.0 up to recently released 5.0.1

% cd /tmp
% git clone https://github.com/nlohmann/json
% cd json
% (mkdir build; cd build; cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ..)
% ln -s build/compile_commands.json
% nvim test/src/unit-constructor1.cpp  # or other editor

If you use cquery --log-file /tmp/cq.log "$@", you'll see

�[0m�[1m�[31m2017-12-25 11:18:29.703 (  43.652s) [indexer0        ]             indexer.cc:1892  WARN| Indexing /tmp/json/test/src/unit-iterators2.cpp failed with errno=1�[0m
libclang: crash detected during indexing TU

errno=1 indicates CXError_Failure which is caught by clang's CrashRecoveryContext (you can add clang_toggleCrashRecovery(0) just before the clang_indexTranslationUnit call to let it crash and see the stack trace in a debugger).

This SIGSEGV is caused by:

https://github.com/llvm-mirror/clang/blob/master/tools/libclang/CXIndexDataConsumer.cpp#L203

    handleReference(ND, Loc, Cursor,
                    dyn_cast_or_null<NamedDecl>(ASTNode.Parent),
                    ASTNode.ContainerDC, ASTNode.OrigE, Kind);

dyn_cast_or_null<NamedDecl>(ASTNode.Parent) is somehow a null pointer and in https://github.com/llvm-mirror/clang/blob/master/tools/libclang/CXIndexDataConsumer.cpp#L935

  ContainerInfo Container;
  getContainerInfo(DC, Container);

The null DC is casted ContInfo.cursor = getCursor(cast<Decl>(DC)); and SIGSEGV.

[topisani]

been having the same issue with nlohmann/json for some time. This diff in libclang "fixes" it, but i have no idea why DC is null in the first place, and i don't have the time to rebuild llvm with debug symbols on my laptop:

Index: CXIndexDataConsumer.cpp
===================================================================
--- CXIndexDataConsumer.cpp	(revision 321065)
+++ CXIndexDataConsumer.cpp	(working copy)
@@ -892,6 +892,8 @@
                                       CXIdxEntityRefKind Kind) {
   if (!D)
     return false;
+  if (!DC)
+    return false;
 
   CXCursor Cursor = E ? MakeCXCursor(E, cast<Decl>(DC), CXTU)
                       : getRefCursor(D, Loc);
@@ -909,6 +911,8 @@
 
   if (!D)
     return false;
+  if (!DC)
+    return false;
   if (Loc.isInvalid())
     return false;
   if (!shouldIndexFunctionLocalSymbols() && isFunctionLocalDecl(D))

[MaskRay]

On Linux, if you have installed radare2 and use ./waf configure --bundled-clang=5.0.1, here is a poor man's fix:

r2 -nwqc 'wx 4d85c0 @ 0x47aece' build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5.0

[MaskRay]

bool CXIndexDataConsumer::handleReference(const NamedDecl *D, SourceLocation Loc,
                                      CXCursor Cursor,
                                      const NamedDecl *Parent,
                                      const DeclContext *DC,
                                      const Expr *E,
                                      CXIdxEntityRefKind Kind);

According to System V x86-64 ABI, its parameters are passed in the following way:

this: rdi
D: rsi
Loc: rdx
Cursor: stack
Parent: rcx
DC: r8
E: r9
Kind: stack

We know CB.indexEntityReference must be non-NULL because cquery sets it to OnIndexReference.

% objdump -Cd build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5 --start-address 0x47ae90 --stop-address 0x47b190
......
  47aeb0:	45 31 ed             	xor    r13d,r13d
  47aeb3:	45 85 ff             	test   r15d,r15d
  47aeb6:	0f 84 19 03 00 00    	je     47b1d5
  47aebc:	48 85 ed             	test   rbp,rbp
  47aebf:	0f 84 10 03 00 00    	je     47b1d5
  47aec5:	49 8b 44 24 18       	mov    rax,QWORD PTR [r12+0x18]  # load `IndexerCallbacks &CB`, which is actually a pointer
  47aeca:	48 8b 40 38          	mov    rax,QWORD PTR [rax+0x38]  # load CB.indexEntityReference
  47aece:	48 85 c0             	test   rax,rax     # a redundant check to see if it is null; we replace it with check of `DC`
  47aed1:	0f 84 fe 02 00 00    	je     47b1d5 <clang::cxindex::CXIndexDataConsumer::handleReference(clang::NamedDecl const*, clang::SourceLocation, CXCursor, clang::NamedDecl const*, clang::DeclContext const*, clang::Expr const*, CXIdxEntityRefKind)+0x345>

test rax,rax at 0x47aeca is redundant and we replace it with test r8,r8, testing whether DC is null.

# 48 85 c0  test rax,rax
# 4d 85 c0  test r8,r8

Which can be achieved with

% r2 -nwqc 'wx 4d @ 0x47aece' build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5.0

or

% printf '\x4d' | dd of=build/release/lib/clang+llvm-5.0.1-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.5.0 obs=1 seek=$[0x47aece] conv=notrunc
% printf '\x4d' | dd of=build/release/lib/clang+llvm-4.0.0-x86_64-linux-gnu-ubuntu-14.04/lib/libclang.so.4.0 obs=1 seek=$[0x4172b5] conv=notrunc