How to collect coverage data when running E2E tests?

[gkorland]

We're building a library that not only being tested by unit tests but also as part E2E tests.

1. When can I find the instrumented build binaries?
   This is how I build the library cargo tarpaulin --no-run

2. How can I collect coverage info as part of the E2E tests?

[xd009642]

So lets say we have a project using assert_cmd for integration testing of binaries and a test like:

use assert_cmd::prelude::*;
use std::process::Command;

#[test]
fn run_integration_test() {
    let mut cmd = Command::cargo_bin("app").unwrap();
    cmd.assert().success();
}

Then we can just do cargo tarpaulin --follow-exec. If you're running your tests some other way then if you describe how you're planning it I can try to advise on how to get the coverage - the solution may involve using tarpaulin.toml to specify getting the coverage in multiple steps, and/or using something like --run-types Bins --bin $BINARY to run a specific application in the project

[lcmgh]

What if integration tests do not include the code for bootstrapping the app but that is done via some other CI commands?

[xd009642]

Then (generally speaking), I would run those commands before running tarpaulin. I'm sure there's likely reasons for some people why that's not possible - but I'd need more specific examples to give more specific examples.

[lcmgh]

Yeah I decided to spawn the necessary processes manually. But from my understanding

let mut cmd = Command::cargo_bin("app").unwrap();
cmd.assert().success();

Is blocking while one normally needs to keep the process running during time of the integration tests.

For that use case https://crates.io/crates/expectrl seems more appropriate.

[xd009642]

Ah right yeah I follow I wasn't really tracking from the context of the prior question.

So my first answer was done a bit probably thinking about CLI apps and people wanting E2E tests for a CLI (a more often requested thing).

For servers, personally I often do a start up and tear down for every e2e test and use a random free port per instance just to ensure each test is an isolated environment. That way I can also test different launch configs, global state etc which I couldn't do with just one running one.

I guess a way to launch one shared server and get coverage for it would be to make some sort of OnceLock type holder to a server handle and then on Drop shutdown the server. That way the coverage instrumentation data gets written out before tarpaulin exits and the results are reported.