Produce code coverage reports for Rust code

[legoktm]

Just like we have pytest output a coverage report, we should have code coverage reports for Rust.

https://www.collabora.com/news-and-blog/blog/2021/03/24/rust-integrating-llvm-source-base-code-coverage-with-gitlab/ is what I based the mwbot-rs pipeline off of https://gitlab.wikimedia.org/repos/mwbot-rs/rust-ci-pipeline/-/blob/main/jobs.yml#L132 and it mostly works.

There's also https://github.com/actions-rs/grcov which is unmaintained/abandoned but some of the issues might have useful pointers as well.

[surfingreg]

I looked at a few different courses of action for this. grcov seems like it was well developed, but yeah, probably abandoned. The default llvm-cov report seems fairly well done. Here's what they look like without a whole lot of trying.

llvm-cov:

grcov:

[surfingreg]

References below but here's the how-to for the llvm version:

# start fresh
find . -name "*.profraw" -exec rm {} \;
rm redwood/*.profraw
rm -Rf target/llvm-cov
cargo uninstall cargo-binutils 
rustup component remove llvm-tools-preview

# one-time installs:
cargo install cargo-binutils
rustup component remove llvm-tools-preview
cargo install cargo-llvm-cov

# run the coverage report:
# output format for intermediate profraw files:
export LLVM_PROFILE_FILE="securedrop-%p-%m.profraw"
# generate the intermediate .profraw files
cargo clean
# RUSTFLAGS="-C instrument-coverage" cargo build
RUSTFLAGS="-C instrument-coverage" cargo test --tests

# generate the report
cargo llvm-cov --html

# view the report
open target/llvm-cov/html

The process is basically the same for grcov...

rustup component add llvm-tools-preview
cargo install grcov
export RUSTFLAGS="-Cinstrument-coverage"
cargo build
export LLVM_PROFILE_FILE="target/grcov/securedrop-%p-%m.profraw"
cargo test --verbose
grcov . -s . --binary-path ./target/debug/ -t html ./target/debug/
[open ./html]

rustc book:
https://doc.rust-lang.org/rustc/instrument-coverage.html#creating-coverage-reports
https://doc.rust-lang.org/rustc/instrument-coverage.html#creating-coverage-reports
https://github.com/rust-embedded/cargo-binutils
https://github.com/taiki-e/cargo-llvm-cov
https://stackoverflow.com/questions/69491669/how-i-can-get-coverage-for-cargo-test

[surfingreg]

So the "todo" would be for me to learn how the Secure Drop CI process works (similar to what your grcov link does), integrate this process into it, and integrate the report into the overall project report. Is that where you all want this to go? I'll look into the securedrop build files and see where an actual PR would come in.