From 7a8cade9c7b78eb07a7ab69d14e605b2060f0f0e Mon Sep 17 00:00:00 2001 From: Ezra Shaw Date: Tue, 4 Apr 2023 23:24:25 +1200 Subject: [PATCH] docs: document new `suggest-tests` tool --- src/SUMMARY.md | 1 + src/building/suggested.md | 17 ++++++++++++ src/tests/suggest-tests.md | 55 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 73 insertions(+) create mode 100644 src/tests/suggest-tests.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 7c85576ff..bdcd3e27d 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -25,6 +25,7 @@ - [Test headers](./tests/headers.md) - [Performance testing](./tests/perf.md) - [Crater](./tests/crater.md) + - [Suggest tests tool](./tests/suggest-tests.md) - [Debugging the compiler](./compiler-debugging.md) - [Using the tracing/logging instrumentation](./tracing.md) - [Profiling the compiler](./profiling.md) diff --git a/src/building/suggested.md b/src/building/suggested.md index 3049d87db..8cf9891b6 100644 --- a/src/building/suggested.md +++ b/src/building/suggested.md @@ -109,6 +109,23 @@ the problem. A nice side-effect of this style is that you are left with a fairly fine-grained set of commits at the end, all of which build and pass tests. This often helps reviewing. +## `x suggest` + +The `x suggest` subcommand suggests (and runs) a subset of the extensive +`rust-lang/rust` tests based on files you have changed. This is especially useful +for new contributors who have not mastered the arcane `x` flags yet and more +experienced contributors as a shorthand for reducing mental effort. In all cases +it is useful not to run the full tests (which can take on the order of tens of +minutes) and just run a subset which are relevant to your changes. For example, +running `tidy` and `linkchecker` is useful when editing Markdown files, whereas UI +tests are much less likely to be helpful. While `x suggest` is a useful tool, it +does not guarantee perfect coverage (just as PR CI isn't a substitute for bors). +See the [dedicated chapter](../tests/suggest-tests.md) for more information and +contribution instructions. + +Please note that `x suggest` is in a beta state currently and the tests that it +will suggest are limited. + ## Configuring `rustup` to use nightly Some parts of the bootstrap process uses pinned, nightly versions of tools like diff --git a/src/tests/suggest-tests.md b/src/tests/suggest-tests.md new file mode 100644 index 000000000..70d48bf70 --- /dev/null +++ b/src/tests/suggest-tests.md @@ -0,0 +1,55 @@ +# Suggest tests tool + +This chapter is about the internals of and contribution instructions for the +`suggest-tests` tool. For a high-level overview of the tool, see +[this section](../building/suggested.md#x-suggest). This tool is currently in a +beta state and is tracked by [this](https://github.com/rust-lang/rust/issues/109933) +issue on Github. Currently the number of tests it will suggest are very limited +in scope, we are looking to expand this (contributions welcome!). + +## Internals + +The tool is defined in a separate crate ([`src/tools/suggest-tests`](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests)) +which outputs suggestions which are parsed by a shim in bootstrap +([`src/bootstrap/suggest.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/suggest.rs)). +The only notable thing the bootstrap shim does is (when invoked with the +`--run` flag) use bootstrap's internal mechanisms to create a new `Builder` and +uses it to invoke the suggested commands. The `suggest-tests` crate is where the +fun happens, two kinds of suggestions are defined: "static" and "dynamic" +suggestions. + +### Static suggestions + +Defined [here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs). +Static suggestions are simple: they are just [globs](https://crates.io/crates/glob) +which map to a `x` command. In `suggest-tests`, this is implemented with a +simple `macro_rules` macro. + +### Dynamic suggestions + +Defined [here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/dynamic_suggestions.rs). +These are more complicated than static suggestions and are implemented as +functions with the following signature: `fn(&Path) -> Vec`. In +other words, each suggestion takes a path to a modified file and (after running +arbitrary Rust code) can return any number of suggestions, or none. Dynamic +suggestions are useful for situations where fine-grained control over +suggestions is needed. For example, modifications to the `compiler/xyz/` path +should trigger the `x test compiler/xyz` suggestion. In the future, dynamic +suggestions might even read file contents to determine if (what) tests should +run. + +## Adding a suggestion + +The following steps should serve as a rough guide to add suggestions to +`suggest-tests` (very welcome!): + +1. Determine the rules for your suggestion. Is it simple and operates only on + a single path or does it match globs? Does it need fine-grained control over + the resulting command or does "one size fit all"? +2. Based on the previous step, decide if your suggestion should be implemented + as either static or dynamic. +3. Implement the suggestion. If it is dynamic then a test is highly recommended, + to verify that your logic is correct and to give an example of the suggestion. + See the [tests.rs](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs) + file. +4. Open a PR implementing your suggestion. **(TODO: add example PR)**