Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for generating unique profraw files by default when using -C instrument-coverage #100384

Merged
merged 3 commits into from
Aug 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions compiler/rustc_codegen_llvm/src/back/write.rs
Original file line number Diff line number Diff line change
Expand Up @@ -423,6 +423,14 @@ fn get_pgo_sample_use_path(config: &ModuleConfig) -> Option<CString> {
.map(|path_buf| CString::new(path_buf.to_string_lossy().as_bytes()).unwrap())
}

fn get_instr_profile_output_path(config: &ModuleConfig) -> Option<CString> {
if config.instrument_coverage {
Some(CString::new("default_%m_%p.profraw").unwrap())
} else {
None
}
}

pub(crate) unsafe fn optimize_with_new_llvm_pass_manager(
cgcx: &CodegenContext<LlvmCodegenBackend>,
diag_handler: &Handler,
Expand All @@ -438,6 +446,7 @@ pub(crate) unsafe fn optimize_with_new_llvm_pass_manager(
let pgo_use_path = get_pgo_use_path(config);
let pgo_sample_use_path = get_pgo_sample_use_path(config);
let is_lto = opt_stage == llvm::OptStage::ThinLTO || opt_stage == llvm::OptStage::FatLTO;
let instr_profile_output_path = get_instr_profile_output_path(config);
// Sanitizer instrumentation is only inserted during the pre-link optimization stage.
let sanitizer_options = if !is_lto {
Some(llvm::SanitizerOptions {
Expand Down Expand Up @@ -488,6 +497,7 @@ pub(crate) unsafe fn optimize_with_new_llvm_pass_manager(
pgo_gen_path.as_ref().map_or(std::ptr::null(), |s| s.as_ptr()),
pgo_use_path.as_ref().map_or(std::ptr::null(), |s| s.as_ptr()),
config.instrument_coverage,
instr_profile_output_path.as_ref().map_or(std::ptr::null(), |s| s.as_ptr()),
config.instrument_gcov,
pgo_sample_use_path.as_ref().map_or(std::ptr::null(), |s| s.as_ptr()),
config.debug_info_for_profiling,
Expand Down
1 change: 1 addition & 0 deletions compiler/rustc_codegen_llvm/src/llvm/ffi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2347,6 +2347,7 @@ extern "C" {
PGOGenPath: *const c_char,
PGOUsePath: *const c_char,
InstrumentCoverage: bool,
InstrProfileOutput: *const c_char,
InstrumentGCOV: bool,
PGOSampleUsePath: *const c_char,
DebugInfoForProfiling: bool,
Expand Down
8 changes: 6 additions & 2 deletions compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -822,7 +822,8 @@ LLVMRustOptimizeWithNewPassManager(
bool DisableSimplifyLibCalls, bool EmitLifetimeMarkers,
LLVMRustSanitizerOptions *SanitizerOptions,
const char *PGOGenPath, const char *PGOUsePath,
bool InstrumentCoverage, bool InstrumentGCOV,
bool InstrumentCoverage, const char *InstrProfileOutput,
bool InstrumentGCOV,
const char *PGOSampleUsePath, bool DebugInfoForProfiling,
void* LlvmSelfProfiler,
LLVMRustSelfProfileBeforePassCallback BeforePassCallback,
Expand Down Expand Up @@ -922,8 +923,11 @@ LLVMRustOptimizeWithNewPassManager(

if (InstrumentCoverage) {
PipelineStartEPCallbacks.push_back(
[](ModulePassManager &MPM, OptimizationLevel Level) {
[InstrProfileOutput](ModulePassManager &MPM, OptimizationLevel Level) {
InstrProfOptions Options;
if (InstrProfileOutput) {
Options.InstrProfileOutput = InstrProfileOutput;
}
MPM.addPass(InstrProfiling(Options, false));
}
);
Expand Down
28 changes: 20 additions & 8 deletions src/doc/rustc/src/instrument-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,17 @@ $ echo "{some: 'thing'}" | target/debug/examples/formatjson5 -
}
```

After running this program, a new file, `default.profraw`, should be in the current working directory. It's often preferable to set a specific file name or path. You can change the output file using the environment variable `LLVM_PROFILE_FILE`:
After running this program, a new file named like `default_11699812450447639123_0_20944` should be in the current working directory.
A new, unique file name will be generated each time the program is run to avoid overwriting previous data.

```shell
$ echo "{some: 'thing'}" | target/debug/examples/formatjson5 -
...
$ ls default_*.profraw
default_11699812450447639123_0_20944.profraw
```

You can also set a specific file name or path for the generated `.profraw` files by using the environment variable `LLVM_PROFILE_FILE`:

```shell
$ echo "{some: 'thing'}" \
Expand All @@ -115,6 +125,9 @@ If `LLVM_PROFILE_FILE` contains a path to a non-existent directory, the missing
- `%Nm` - the instrumented binary’s signature: The runtime creates a pool of N raw profiles, used for on-line profile merging. The runtime takes care of selecting a raw profile from the pool, locking it, and updating it before the program exits. `N` must be between `1` and `9`, and defaults to `1` if omitted (with simply `%m`).
- `%c` - Does not add anything to the filename, but enables a mode (on some platforms, including Darwin) in which profile counter updates are continuously synced to a file. This means that if the instrumented program crashes, or is killed by a signal, perfect coverage information can still be recovered.

In the first example above, the value `11699812450447639123_0` in the generated filename is the instrumented binary's signature,
which replaced the `%m` pattern and the value `20944` is the process ID of the binary being executed.

## Installing LLVM coverage tools

LLVM's supplies two tools—`llvm-profdata` and `llvm-cov`—that process coverage data and generate reports. There are several ways to find and/or install these tools, but note that the coverage mapping data generated by the Rust compiler requires LLVM version 12 or higher, and processing the *raw* data may require exactly the LLVM version used by the compiler. (`llvm-cov --version` typically shows the tool's LLVM version number, and `rustc --verbose --version` shows the version of LLVM used by the Rust compiler.)
Expand Down Expand Up @@ -181,11 +194,10 @@ A typical use case for coverage analysis is test coverage. Rust's source-based c

The following example (using the [`json5format`] crate, for demonstration purposes) show how to generate and analyze coverage results for all tests in a crate.

Since `cargo test` both builds and runs the tests, we set both the additional `RUSTFLAGS`, to add the `-C instrument-coverage` flag, and `LLVM_PROFILE_FILE`, to set a custom filename for the raw profiling data generated during the test runs. Since there may be more than one test binary, apply `%m` in the filename pattern. This generates unique names for each test binary. (Otherwise, each executed test binary would overwrite the coverage results from the previous binary.)
Since `cargo test` both builds and runs the tests, we set the additional `RUSTFLAGS`, to add the `-C instrument-coverage` flag.

```shell
$ RUSTFLAGS="-C instrument-coverage" \
LLVM_PROFILE_FILE="json5format-%m.profraw" \
cargo test --tests
```

Expand All @@ -210,7 +222,7 @@ test result: ok. 31 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
You should have one or more `.profraw` files now, one for each test binary. Run the `profdata` tool to merge them:

```shell
$ llvm-profdata merge -sparse json5format-*.profraw -o json5format.profdata
$ llvm-profdata merge -sparse default_*.profraw -o json5format.profdata
```

Then run the `cov` tool, with the `profdata` file and all test binaries:
Expand All @@ -230,6 +242,8 @@ $ llvm-cov show \
--Xdemangler=rustfilt | less -R
```

> **Note**: If overriding the default `profraw` file name via the `LLVM_PROFILE_FILE` environment variable, it's highly recommended to use the `%m` and `%p` special pattern strings to generate unique file names in the case of more than a single test binary being executed.

> **Note**: The command line option `--ignore-filename-regex=/.cargo/registry`, which excludes the sources for dependencies from the coverage results.\_

### Tips for listing the binaries automatically
Expand Down Expand Up @@ -271,9 +285,8 @@ To include doc tests in the coverage results, drop the `--tests` flag, and apply
```bash
$ RUSTFLAGS="-C instrument-coverage" \
RUSTDOCFLAGS="-C instrument-coverage -Z unstable-options --persist-doctests target/debug/doctestbins" \
LLVM_PROFILE_FILE="json5format-%m.profraw" \
cargo test
$ llvm-profdata merge -sparse json5format-*.profraw -o json5format.profdata
$ llvm-profdata merge -sparse default_*.profraw -o json5format.profdata
```

The `-Z unstable-options --persist-doctests` flag is required, to save the test binaries
Expand Down Expand Up @@ -302,8 +315,7 @@ $ llvm-cov report \
> version without doc tests, include:

- The `cargo test ... --no-run` command is updated with the same environment variables
and flags used to _build_ the tests, _including_ the doc tests. (`LLVM_PROFILE_FILE`
is only used when _running_ the tests.)
and flags used to _build_ the tests, _including_ the doc tests.
- The file glob pattern `target/debug/doctestbins/*/rust_out` adds the `rust_out`
binaries generated for doc tests (note, however, that some `rust_out` files may not
be executable binaries).
Expand Down
17 changes: 17 additions & 0 deletions src/test/codegen/instrument-coverage.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Test that `-Cinstrument-coverage` creates expected __llvm_profile_filename symbol in LLVM IR.

// needs-profiler-support
// compile-flags: -Cinstrument-coverage

// CHECK: @__llvm_profile_filename = {{.*}}"default_%m_%p.profraw\00"{{.*}}

#![crate_type="lib"]

#[inline(never)]
fn some_function() {

}

pub fn some_other_function() {
some_function();
}