Sccache claims "multiple input files" for Rust compile when parsing --check-cfg

[cr1901]

I have in the past used sccache to speed up building the rust compiler when I need to bisect issues. Yesterday, I noticed that sccache has stopped (or perhaps never cached in the first place? I can't tell...) caching crates from the rust-lang compiler/rustc workspace members. sccache -s reports the following:

william@xubuntu-dtrain:~/Projects/toolchains/build-llvm-toolchain/build-rust$ sccache -s
Compile requests                      5
Compile requests executed             0
Cache hits                            0
Cache misses                          0
Cache timeouts                        0
Cache read errors                     0
Forced recaches                       0
Cache write errors                    0
Compilation failures                  0
Cache errors                          0
Non-cacheable compilations            0
Non-cacheable calls                   5
Non-compilation calls                 0
Unsupported compiler calls            0
Average cache write               0.000 s
Average cache read miss           0.000 s
Average cache read hit            0.000 s
Failed distributed compilations       0

Non-cacheable reasons:
-                                     4
multiple input files                  1

Cache location                  Local disk: "/home/william/.cache/sccache"

Although "multiple input files" only appears once, this is because I stopped the build on the first crate that needed a compile. I produced the above using the following command line. _Everything before the ... was reported "Fresh" by cargo, so I omitted it. "Multiple input files" would be reported for subsequent crates if I let the build continue.

william@xubuntu-dtrain:~/Projects/toolchains/build-llvm-toolchain/build-rust$ RUSTC_WRAPPER=sccache /home/william/Projects/toolchains/rust/x.py -v build -j1 --config config.toml --stage 1 library/test
...
Compiling rustc_middle v0.0.0 (/home/william/Projects/toolchains/rust/compiler/rustc_middle)
     Running `sccache /home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/bootstrap/debug/rustc --crate-name rustc_middle --edition=2021 compiler/rustc_middle/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C debuginfo=1 -Zunstable-options --check-cfg 'values(feature, "rustc-rayon", "rustc-rayon-core", "rustc_use_parallel_compiler")' --check-cfg 'names()' --check-cfg 'values()' -C metadata=78f997ffff71ccf5 -C extra-filename=-78f997ffff71ccf5 --out-dir /home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps -L dependency=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps --extern bitflags=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libbitflags-4c86917066846c89.rmeta --extern chalk_ir=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libchalk_ir-205491956786ecdf.rmeta --extern either=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libeither-52b96456516b2d41.rmeta --extern gsgdt=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libgsgdt-06323408899e8337.rmeta --extern polonius_engine=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libpolonius_engine-faf308848992afcf.rmeta --extern rand=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librand-073b9756c31adb57.rmeta --extern rand_xoshiro=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librand_xoshiro-4a5c267b4f405eaa.rmeta --extern rustc_apfloat=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_apfloat-8c1bdd142481172f.rmeta --extern rustc_arena=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_arena-2ec1b8415aeb8a0d.rmeta --extern rustc_ast=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_ast-6f1c0f30a9110dbb.rmeta --extern rustc_attr=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_attr-bb3250f5147257ec.rmeta --extern rustc_data_structures=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_data_structures-90ccfdb92ef57dc3.rmeta --extern rustc_errors=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_errors-c33dc17a64be6f43.rmeta --extern rustc_feature=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_feature-794599351f8ce486.rmeta --extern rustc_graphviz=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_graphviz-0a01a8f25186c87e.rmeta --extern rustc_hir=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_hir-a66253c291db6170.rmeta --extern rustc_index=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_index-4c1798ace25baa1f.rmeta --extern rustc_macros=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/librustc_macros-6600c0a1fc12bee5.so --extern rustc_query_system=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_query_system-3f89ce43252a0f0b.rmeta --extern rustc_serialize=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_serialize-f1a8bb37c6aca652.rmeta --extern rustc_session=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_session-fb9d8f4d8c724a42.rmeta --extern rustc_span=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_span-e05417af35840254.rmeta --extern rustc_target=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_target-86ec3ec3d1d32b71.rmeta --extern rustc_type_ir=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_type_ir-425f2e60df76099a.rmeta --extern smallvec=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libsmallvec-90d066a285d5ff14.rmeta --extern tracing=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libtracing-af24c95a9b28d715.rmeta --cfg=bootstrap -Csymbol-mangling-version=v0 -Zunstable-options '--check-cfg=values(bootstrap)' '--check-cfg=values(parallel_compiler)' '--check-cfg=values(no_btreemap_remove_entry)' '--check-cfg=values(crossbeam_loom)' '--check-cfg=values(span_locations)' -Zmacro-backtrace -Clink-args=-Wl,-z,origin '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Zunstable-options -Csplit-debuginfo=off -Ztls-model=initial-exec -Zunstable-options '-Wrustc::internal' -Cprefer-dynamic -Z binary-dep-depinfo -L native=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/psm-87f8634215325b47/out`

If I start a gdb session for sccache and break on the line that displays the "multiple input files" string, I see the following:

(gdb) p *val
$1 = std::ffi::os_str::OsString {inner: std::sys::unix::os_str::Buf {inner: alloc::vec::Vec<u8, alloc::alloc::Global> {buf: alloc::raw_vec::RawVec<u8, alloc::alloc::Global> {ptr: core::ptr::unique::Unique<u8> {pointer: core::ptr::non_null::NonNull<u8> {pointer: 0x7fffa000f6f0 "values(feature, \"rustc-rayon\", \"rustc-rayon-core\", \"rustc_use_parallel_compiler\")rnings\000"}, _marker: core::marker::PhantomData<u8>}, cap: 81, alloc: alloc::alloc::Global}, len: 81}}}

values(feature, "rustc-rayon", "rustc-rayon-core", "rustc_use_parallel_compiler") is not a Raw argument; it is the input to the recently-added --check-cfg option.

I'm assuming the problem is that sccache does not understand the recently-added --check-cfg option yet, and when support is added for this option, sccache will correctly identify the above command line as having a single input file.

[cr1901]

Hmmm, this did not fix the issue for me. Using the same setup as last time:

william@xubuntu-dtrain:~/Projects/toolchains/build-llvm-toolchain/build-rust$ RUSTC_WRAPPER=sccache ../../rust/x.py -v build -j1 --config config.toml --stage 1 library/test
...
Compiling rustc_middle v0.0.0 (/home/william/Projects/toolchains/rust/compiler/rustc_middle)
  Running `sccache /home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/bootstrap/debug/rustc --crate-name rustc_middle --edition=2021 compiler/rustc_middle/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C debuginfo=1 -Zunstable-options --check-cfg 'values(feature, "rustc-rayon", "rustc-rayon-core", "rustc_use_parallel_compiler")' --check-cfg 'names()' --check-cfg 'values()' -C metadata=7e4d058ba8dc89c2 -C extra-filename=-7e4d058ba8dc89c2 --out-dir /home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps -L dependency=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps --extern bitflags=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libbitflags-ae63c3954d1ecfc2.rmeta --extern chalk_ir=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libchalk_ir-e36ee0691f5e4dd4.rmeta --extern either=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libeither-52b96456516b2d41.rmeta --extern gsgdt=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libgsgdt-1f5adbaa9051b72c.rmeta --extern polonius_engine=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libpolonius_engine-faf308848992afcf.rmeta --extern rand=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librand-8cf822826caee2fc.rmeta --extern rand_xoshiro=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librand_xoshiro-420f8881ecfe8802.rmeta --extern rustc_apfloat=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_apfloat-9b8a4323fe7cd3c1.rmeta --extern rustc_arena=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_arena-2ec1b8415aeb8a0d.rmeta --extern rustc_ast=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_ast-7d40d76232c5d2df.rmeta --extern rustc_attr=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_attr-05d818a3da9833f6.rmeta --extern rustc_data_structures=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_data_structures-18a4d81ad070020a.rmeta --extern rustc_errors=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_errors-f721b87d0ea82d1b.rmeta --extern rustc_feature=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_feature-316b4d0eb69aeb7f.rmeta --extern rustc_graphviz=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_graphviz-0a01a8f25186c87e.rmeta --extern rustc_hir=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_hir-a513aa67a44bda88.rmeta --extern rustc_index=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_index-ab36757e4a74ace1.rmeta --extern rustc_macros=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps/librustc_macros-d62456eac85037a2.so --extern rustc_query_system=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_query_system-0d3916a27296b2c5.rmeta --extern rustc_serialize=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_serialize-ff661c02782029ee.rmeta --extern rustc_session=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_session-df2afd48c3843112.rmeta --extern rustc_span=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_span-4d387ed68540f222.rmeta --extern rustc_target=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_target-3babe06db02e766f.rmeta --extern rustc_type_ir=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_type_ir-c9e63595e144177c.rmeta --extern smallvec=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libsmallvec-90d066a285d5ff14.rmeta --extern thin_vec=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libthin_vec-d3e80878b1f97df9.rmeta --extern tracing=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libtracing-ceb0d931525cc659.rmeta --cfg=bootstrap -Csymbol-mangling-version=v0 -Zunstable-options '--check-cfg=values(bootstrap)' '--check-cfg=values(parallel_compiler)' '--check-cfg=values(no_btreemap_remove_entry)' '--check-cfg=values(crossbeam_loom)' '--check-cfg=values(span_locations)' -Zmacro-backtrace -Clink-args=-Wl,-z,origin '-Clink-args=-Wl,-rpath,$ORIGIN/../lib' -Csplit-debuginfo=off -Zunstable-options '-Wrustc::internal' -Cprefer-dynamic -Z binary-dep-depinfo -L native=/home/william/Projects/toolchains/build-llvm-toolchain/build-rust/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/psm-6b16bd2d4bbe363b/out`

sccache still gets stuck thinking rustc crates have multiple input files. The string changed however; now sccache thinks a single file compiler/rustc_middle/src/lib.rs is multiple files :P:

Thread 20 "tokio-runtime-w" hit Breakpoint 1, sccache::compiler::rust::parse_arguments (
    arguments=..., cwd=...) at src/compiler/rust.rs:1131
1131                            if input.is_some() {
(gdb) p *val
$1 = std::ffi::os_str::OsString {inner: std::sys::unix::os_str::Buf {inner: alloc::vec::Vec<u8, alloc::alloc::Global> {buf: alloc::raw_vec::RawVec<u8, alloc::alloc::Global> {ptr: core::ptr::unique::Unique<u8> {pointer: core::ptr::non_null::NonNull<u8> {pointer: 0x7fffac004a30 "compiler/rustc_middle/src/lib.rs\340\005\000"}, _marker: core::marker::PhantomData<u8>}, cap: 32, alloc: alloc::alloc::Global}, len: 32}}}

[drahnr]

Could you create a unit test to verify your exact case, how it'd be parsed?

[cr1901]

@drahnr I can't duplicate the problem anymore- caching is working fine now. I think what happened was two things:

1. I forgot to kill the previously running sccache instance after installing the new binary.
2. When I went to debug, thinking the new binary still has the issue, I forgot to run cargo build, and immediately invoked gdb on a stale debug binary that still had the issue.

Anyways, the problem is fixed, and since I can't dupe, my previous message can be ignored.

[drahnr]

Ok, thanks for re-cheching!