Path Interaction Between show-env, nextest, and report

[cwfitzgerald]

I'm not quite sure what exactly is happening here, but here is the general flow I'm following:

## Build Machine Job 1 ##
source <(cargo llvm-cov show-env --export-prefix --no-cfg-coverage)
cargo build --bin wgpu-info --target ${{ matrix.target }}

## Build Machine Job 2 (env is reset) ##
cargo llvm-cov nextest-archive --all-features --no-cfg-coverage --archive-file tests.tar.zst --target ${{ matrix.target }}

## Transfer Binaries, Archive, etc ##

## Test Machine Job 1 ##
    # This has LLVM_PROFILE_FILE point to <snip>/target/wgpu-%p-%20m.profraw
source <(cargo llvm-cov show-env --export-prefix --no-cfg-coverage)

    # This command puts profraws in `target/` due to the above
./wgpu-info 

    # This seems to expect profraws in `target/debug`, and hence finds nothing and errors.
cargo llvm-cov report --target ${{ matrix.target }} --lcov --output-path wgpu-info-lcov.info 

## Test Machine Job 2 ##
    # These all work fine now!
cargo llvm-cov clean
cargo llvm-cov --output-path tests-lcov.info nextest --archive-file tests.tar.zst --extract-overwrite  --target ${{ matrix.target }}

The real CI config I'm running: build machine test machine

Aside

In my ideal world, I would be able to combine both the profraws from both the nextest and the wgpu-info run into a single lcov profile. I can do this if I'm running locally, but I don't think report would pick up on the nextest profraws.

I can live with two profraws - codecov lets you upload multiple files - would be nice though :)

[taiki-e]

# This seems to expect profraws in `target/debug`, and hence finds nothing and errors.

It expects both profraws and objects (binaries) to be in the target directory. Try putting the binary in the directory the same as the original location.

[cwfitzgerald]

So I tried two different permutations:

Two Reports

source <(cargo llvm-cov show-env --export-prefix --no-cfg-coverage)

# This needs to match the command in xtask/tests.rs
chmod +x ./target/${{ matrix.target }}/debug/wgpu-info${{ matrix.suffix }}
./target/${{ matrix.target }}/debug/wgpu-info${{ matrix.suffix }} -vv
./target/${{ matrix.target }}/debug/wgpu-info${{ matrix.suffix }} --json -o .gpuconfig

cargo llvm-cov report --target ${{ matrix.target }} --lcov --output-path wgpu-info-lcov.info

## Job 2

cargo llvm-cov --lcov --output-path tests-lcov.info nextest --archive-file tests.tar.zst --extract-overwrite --no-fail-fast --retries 2 --target ${{ matrix.target }}

This worked fine for the windows and linux jobs, but the llvm-cov report returned the following on arm mac, after cross compiling

info: when --target option is used, coverage for proc-macro and build script will not be displayed because cargo does not pass RUSTFLAGS to them
warning: not found *.profraw files in /Users/runner/work/wgpu/wgpu/target/llvm-cov-target; this may occur if target directory is accidentally cleared, or running report subcommand without running any tests or binaries
error: no input files specified. See llvm-profdata merge -help
error: failed to merge profile data: process didn't exit successfully: `/Users/runner/.rustup/toolchains/1.71-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin/llvm-profdata merge -sparse -f /Users/runner/work/wgpu/wgpu/target/llvm-cov-target/wgpu-profraw-list -o /Users/runner/work/wgpu/wgpu/target/llvm-cov-target/wgpu.profdata` (exit status: 1)
Error: Process completed with exit code 1.

Full log here

One Report

I also tried combining the nextest and info reports into one report by passing --no-report then calling

cargo llvm-cov report --target ${{ matrix.target }} --lcov --output-path lcov.info

This fails with the same error on all platforms

info: when --target option is used, coverage for proc-macro and build script will not be displayed because cargo does not pass RUSTFLAGS to them
warning: not found object files (searched directories: D:\a\wgpu\wgpu\target\llvm-cov-target\x86_64-pc-windows-msvc\debug); this may occur if show-env subcommand is used incorrectly (see docs or other warnings), or unsupported commands such as nextest archive are used
error: failed to generate report: process didn't exit successfully: `'C:\Users\runneradmin\.rustup\toolchains\1.71-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\bin\llvm-cov.exe' export -format=lcov '-instr-profile=D:\a\wgpu\wgpu\target\llvm-cov-target\wgpu.profdata' -ignore-filename-regex '\\rustc\\([0-9a-f]+|[0-9]+\.[0-9]+\.[0-9]+)\\|^D:\\a\\wgpu\\wgpu(\\.*)?\\(tests|examples|benches)\\|^D:\\a\\wgpu\\wgpu\\target\\llvm\-cov\-target($|\\)|^C:\\Users\\runneradmin\\\.cargo\\(registry|git)\\|^C:\\Users\\runneradmin\\\.rustup\\toolchains($|\\)'` (exit code: 1)
--- stderr
No filenames specified!

[taiki-e]

As for the issue about cargo llvm-cov report, I think the option added in #355 is needed.

The other may have been solved in #349.