Skip to content

docs: document new suggest-tests tool #1660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
- [Test headers](./tests/headers.md)
- [Performance testing](./tests/perf.md)
- [Crater](./tests/crater.md)
- [Suggest tests tool](./tests/suggest-tests.md)
- [Debugging the compiler](./compiler-debugging.md)
- [Using the tracing/logging instrumentation](./tracing.md)
- [Profiling the compiler](./profiling.md)
Expand Down
17 changes: 17 additions & 0 deletions src/building/suggested.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,23 @@ the problem. A nice side-effect of this style is that you are left
with a fairly fine-grained set of commits at the end, all of which
build and pass tests. This often helps reviewing.

## `x suggest`

The `x suggest` subcommand suggests (and runs) a subset of the extensive
`rust-lang/rust` tests based on files you have changed. This is especially useful
for new contributors who have not mastered the arcane `x` flags yet and more
experienced contributors as a shorthand for reducing mental effort. In all cases
it is useful not to run the full tests (which can take on the order of tens of
minutes) and just run a subset which are relevant to your changes. For example,
running `tidy` and `linkchecker` is useful when editing Markdown files, whereas UI
tests are much less likely to be helpful. While `x suggest` is a useful tool, it
does not guarantee perfect coverage (just as PR CI isn't a substitute for bors).
See the [dedicated chapter](../tests/suggest-tests.md) for more information and
contribution instructions.

Please note that `x suggest` is in a beta state currently and the tests that it
will suggest are limited.

## Configuring `rustup` to use nightly

Some parts of the bootstrap process uses pinned, nightly versions of tools like
Expand Down
55 changes: 55 additions & 0 deletions src/tests/suggest-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Suggest tests tool

This chapter is about the internals of and contribution instructions for the
`suggest-tests` tool. For a high-level overview of the tool, see
[this section](../building/suggested.md#x-suggest). This tool is currently in a
beta state and is tracked by [this](https://github.com/rust-lang/rust/issues/109933)
issue on Github. Currently the number of tests it will suggest are very limited
in scope, we are looking to expand this (contributions welcome!).

## Internals

The tool is defined in a separate crate ([`src/tools/suggest-tests`](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests))
which outputs suggestions which are parsed by a shim in bootstrap
([`src/bootstrap/suggest.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/suggest.rs)).
The only notable thing the bootstrap shim does is (when invoked with the
`--run` flag) use bootstrap's internal mechanisms to create a new `Builder` and
uses it to invoke the suggested commands. The `suggest-tests` crate is where the
fun happens, two kinds of suggestions are defined: "static" and "dynamic"
suggestions.

### Static suggestions

Defined [here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs).
Static suggestions are simple: they are just [globs](https://crates.io/crates/glob)
which map to a `x` command. In `suggest-tests`, this is implemented with a
simple `macro_rules` macro.

### Dynamic suggestions

Defined [here](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/dynamic_suggestions.rs).
These are more complicated than static suggestions and are implemented as
functions with the following signature: `fn(&Path) -> Vec<Suggestion>`. In
other words, each suggestion takes a path to a modified file and (after running
arbitrary Rust code) can return any number of suggestions, or none. Dynamic
suggestions are useful for situations where fine-grained control over
suggestions is needed. For example, modifications to the `compiler/xyz/` path
should trigger the `x test compiler/xyz` suggestion. In the future, dynamic
suggestions might even read file contents to determine if (what) tests should
run.

## Adding a suggestion

The following steps should serve as a rough guide to add suggestions to
`suggest-tests` (very welcome!):

1. Determine the rules for your suggestion. Is it simple and operates only on
a single path or does it match globs? Does it need fine-grained control over
the resulting command or does "one size fit all"?
2. Based on the previous step, decide if your suggestion should be implemented
as either static or dynamic.
3. Implement the suggestion. If it is dynamic then a test is highly recommended,
to verify that your logic is correct and to give an example of the suggestion.
See the [tests.rs](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
See the [tests.rs](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/static_suggestions.rs)
See the [tests.rs](https://github.com/rust-lang/rust/blob/master/src/tools/suggest-tests/src/tests.rs)

file.
4. Open a PR implementing your suggestion. **(TODO: add example PR)**