Combining compile & link with Clang results in erroneous entry

[roubert]

Create any trivial Hello, World! style program and combine compile & link on the command line like this:

$ g++-13 -o hello hello.cc
$ clang++-17 -o hello hello.cc

That's a perfectly normal thing to do and should work perfectly fine.

Then do that through Bear:

$ bear -- g++-13 -o hello hello.cc

That works fine and generates a compile_commands.json file with a single entry, which contains exactly that one single command line, just as expected.

Then do that with Clang instead:

$ bear -- clang++-17 -o hello hello.cc

That generates a compile_commands.json file with two different entries, one with that one single command line plus one for the underlying execution of the compile step, with a very long arguments list.

This is a problem because that internal command line for the internal compile step can't be executed directly from the command line and trying to use this compilation database with Clang-Tidy will therefore result in a compilation error:

$ clang-tidy-17 -checks='*' hello.cc
[…]
error: error reading 'pic': No such file or directory [clang-diagnostic-error]

That pic comes from the internal command line, which shouldn't have been included in the compile_commands.json file.

This happens both with the Bear 3.1.3 release and with Bear built from source at current HEAD.

[rizsotto]

Hey @roubert , thanks for the report.

Clang has a "driver" which implements the GCC command line, and it calls itself with the more verbose command line flags. Is this the issue? (It means the second entry is not the linking, but the same compiling pass.)

Could you send me the output?

[roubert]

compile_commands.json

[roubert]

Have you had the chance to look into this and can you confirm that it's an issue with Bear (that would be useful to fix)?

[bkryza]

@rizsotto @roubert I think I stumbled on the same issue, steps to reproduce on my side:

Makefile:

hello: src/hello.cc src/hello.h
                clang++-19 -std=c++17 -o hello src/hello.cc

src/hello.cc:

#include <iostream>

#include "hello.h"

auto main() -> int {
    Hello h;
    std::cout << h.hello() << '\n';
    return 0;
}

src/hello.h:

#ifndef __HELLO_H__
#define __HELLO_H__

#include <string_view>

struct Hello {
    auto hello() const -> std::string_view { return "Hello, world!"; }
};

#endif

$ bear -- make hello

$ clang-tidy-19 
4181 warnings generated.
8362 warnings and 1 error generated.
Error while processing /home/bartek/devel/compile_commands_json_gallery/make_bear/src/hello.cc.
error: error reading 'pic': No such file or directory [clang-diagnostic-error]
Suppressed 8362 warnings (8362 in non-user code).
Use -header-filter=.* to display errors from all non-system headers. Use -system-headers to display errors from system headers as well.
Found compiler error(s).

$ cat compile_commands.json
[
  {
    "arguments": [
      "/usr/bin/clang++-19",
      "-c",
      "-std=c++17",
      "-o",
      "hello",
      "src/hello.cc"
    ],
    "directory": "/home/bartek/devel/compile_commands_json_gallery/make_bear",
    "file": "/home/bartek/devel/compile_commands_json_gallery/make_bear/src/hello.cc",
    "output": "/home/bartek/devel/compile_commands_json_gallery/make_bear/hello"
  },
  {
    "arguments": [
      "/usr/lib/llvm-19/bin/clang",
      "-cc1",
      "-triple",
      "x86_64-pc-linux-gnu",
      "-emit-obj",
      "-dumpdir",
      "hello-",
      "-disable-free",
      "-clear-ast-before-backend",
      "-disable-llvm-verifier",
      "-discard-value-names",
      "-main-file-name",
      "-mrelocation-model",
      "pic",
      "-pic-level",
      "2",
      "-pic-is-pie",
      "-mframe-pointer=all",
      "-fmath-errno",
      "-ffp-contract=on",
      "-fno-rounding-math",
      "-mconstructor-aliases",
      "-funwind-tables=2",
      "-target-cpu",
      "x86-64",
      "-tune-cpu",
      "generic",
      "-debugger-tuning=gdb",
      "-fdebug-compilation-dir=/home/bartek/devel/compile_commands_json_gallery/make_bear",
      "-fcoverage-compilation-dir=/home/bartek/devel/compile_commands_json_gallery/make_bear",
      "-resource-dir",
      "/usr/lib/llvm-19/lib/clang/19",
      "-internal-isystem",
      "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14",
      "-internal-isystem",
      "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/x86_64-linux-gnu/c++/14",
      "-internal-isystem",
      "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/backward",
      "-internal-isystem",
      "/usr/lib/llvm-19/lib/clang/19/include",
      "-internal-isystem",
      "/usr/local/include",
      "-internal-isystem",
      "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../x86_64-linux-gnu/include",
      "-internal-externc-isystem",
      "/usr/include/x86_64-linux-gnu",
      "-internal-externc-isystem",
      "/include",
      "-internal-externc-isystem",
      "/usr/include",
      "-std=c++17",
      "-fdeprecated-macro",
      "-ferror-limit",
      "19",
      "-fgnuc-version=4.2.1",
      "-fskip-odr-check-in-gmf",
      "-fcxx-exceptions",
      "-fexceptions",
      "-fcolor-diagnostics",
      "-faddrsig",
      "-D__GCC_HAVE_DWARF2_CFI_ASM=1",
      "-x",
      "c++",
      "-o",
      "/tmp/hello-883047.o",
      "src/hello.cc"
    ],
    "directory": "/home/bartek/devel/compile_commands_json_gallery/make_bear",
    "file": "/home/bartek/devel/compile_commands_json_gallery/make_bear/src/hello.cc",
    "output": "/tmp/hello-883047.o"
  }
]

The offending flag in the arguments array is:

      "-mrelocation-model",
      "pic",

removing it fixes the problem.

My question is why in this case there are 2 entries for src/hello.cc in the compile_commands.json - I assume it's related to the fact that the first entry is the output make and the second from the clang++-19?

If I change clang++-19 to g++ it works fine.

Also when I run clang++-19 in verbose mode I can see this flag in the output:

❯ clang++-19 -std=c++17 -o hello src/hello.cc -###
Ubuntu clang version 19.1.5 (++20241125105251+086d8e6bb5da-1~exp1~20241125105305.67)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-19/bin
 "/usr/lib/llvm-19/bin/clang" "-cc1" "-triple" "x86_64-pc-linux-gnu" "-emit-obj" "-dumpdir" "hello-" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "hello.cc" "-mrelocation-model" "pic" "-pic-level" "2" "-pic-is-pie" "-mframe-pointer=all" "-fmath-errno" "-ffp-contract=on" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=2" "-target-cpu" "x86-64" "-tune-cpu" "generic" "-debugger-tuning=gdb" "-fdebug-compilation-dir=/home/bartek/devel/compile_commands_json_gallery/make_bear" "-fcoverage-compilation-dir=/home/bartek/devel/compile_commands_json_gallery/make_bear" "-resource-dir" "/usr/lib/llvm-19/lib/clang/19" "-internal-isystem" "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14" "-internal-isystem" "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/x86_64-linux-gnu/c++/14" "-internal-isystem" "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/backward" "-internal-isystem" "/usr/lib/llvm-19/lib/clang/19/include" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/lib/gcc/x86_64-linux-gnu/14/../../../../x86_64-linux-gnu/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-std=c++17" "-fdeprecated-macro" "-ferror-limit" "19" "-fgnuc-version=4.2.1" "-fskip-odr-check-in-gmf" "-fcxx-exceptions" "-fexceptions" "-fcolor-diagnostics" "-faddrsig" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "/tmp/hello-10eec2.o" "-x" "c++" "src/hello.cc"
 "/usr/bin/ld" "-z" "relro" "--hash-style=gnu" "--build-id" "--eh-frame-hdr" "-m" "elf_x86_64" "-pie" "-dynamic-linker" "/lib64/ld-linux-x86-64.so.2" "-o" "hello" "/lib/x86_64-linux-gnu/Scrt1.o" "/lib/x86_64-linux-gnu/crti.o" "/usr/lib/gcc/x86_64-linux-gnu/14/crtbeginS.o" "-L/usr/lib/gcc/x86_64-linux-gnu/14" "-L/usr/lib/gcc/x86_64-linux-gnu/14/../../../../lib64" "-L/lib/x86_64-linux-gnu" "-L/lib/../lib64" "-L/usr/lib/x86_64-linux-gnu" "-L/usr/lib/../lib64" "-L/lib" "-L/usr/lib" "/tmp/hello-10eec2.o" "-lstdc++" "-lm" "-lgcc_s" "-lgcc" "-lc" "-lgcc_s" "-lgcc" "/usr/lib/gcc/x86_64-linux-gnu/14/crtendS.o" "/lib/x86_64-linux-gnu/crtn.o"

[rizsotto]

Yes, it's a known issue. The Clang compiler executes itself to do the real compilation, the first execution assemble a different set of arguments. Bear captures both Clang calls (when you call it, and when itself calls it) and that produces two entries in the output.

As a workaround please specify a config file that has compilers_to_exclude where you filter out the second call.

[bkryza]

@rizsotto Thanks, I can confirm that adding the following config:

{
  "compilation": {
    "compilers_to_exclude": ["/usr/lib/llvm-19/bin/clang"]
  }
}

solves the issue.

[roubert]

OK, I've now verified that the compilers_to_exclude workaround works and resolves my immediate problem, thank you.

But it's rather brittle, requiring the list of paths to the compiler to be kept up-to-date and it'd also be possible to invoke the compiler through those paths for any other reason, so it'd be much more convenient if the compiler invoking itself could be detected and filtered out automatically.