From dafdbbbcce9c5496c69e5b7eb9fabd5dd3aa3c12 Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Wed, 5 Jan 2022 13:59:15 +1100 Subject: [PATCH 1/3] Streamline "Getting Started" some more. This is a follow-up to #1279. The "Getting Started" chapter is, TBH, pretty bad when it comes to the stuff about building and testing. It has far too much detail and lots of repetition, which would be overwhelming to a newcomer. This commit removes most of it, leaving behind just quick mentions of the most common `x.py` commands: `check`, `build`, `test`, `fmt`, with links to the appropriate chapters for details. There were a few interesting details that weren't covered elsewhere, so I moved those into other chapters. --- src/building/how-to-build-and-run.md | 23 ++-- src/conventions.md | 22 ++-- src/getting-started.md | 174 +++------------------------ src/rustdoc.md | 10 +- src/tests/running.md | 11 ++ 5 files changed, 58 insertions(+), 182 deletions(-) diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index 4fe1da6d0..339662780 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -125,8 +125,8 @@ Options: -h, --help print this help message ``` -For hacking, often building the stage 1 compiler is enough, but for -final testing and release, the stage 2 compiler is used. +For hacking, often building the stage 1 compiler is enough, which saves a lot +of time. But for final testing and release, the stage 2 compiler is used. `./x.py check` is really fast to build the rust compiler. It is, in particular, very useful when you're doing some kind of @@ -168,7 +168,7 @@ there is a (hacky) workaround. See [the section on "recommended workflows"](./suggested.md) below. Note that this whole command just gives you a subset of the full `rustc` -build. The **full** `rustc` build (what you get if you say `./x.py build +build. The **full** `rustc` build (what you get with `./x.py build --stage 2 compiler/rustc`) has quite a few more steps: - Build `rustc` with the stage1 compiler. @@ -176,20 +176,16 @@ build. The **full** `rustc` build (what you get if you say `./x.py build - Build `std` with stage2 compiler. - Build `librustdoc` and a bunch of other things with the stage2 compiler. - +You almost never need to do this. ## Build specific components -- Build only the core library +If you are working on the standard library, you probably don't need to build +the compiler unless you are planning to use a recently added nightly feature. +Instead, you can just build stage 0, which uses the current beta compiler. ```bash -./x.py build --stage 0 library/core -``` - -- Build only the core and `proc_macro` libraries - -```bash -./x.py build --stage 0 library/core library/proc_macro +./x.py build --stage 0 library/std ``` Sometimes you might just want to test if the part you’re working on can @@ -245,7 +241,8 @@ in other sections: - Building things: - `./x.py build` – builds everything using the stage 1 compiler, not just up to `std` - - `./x.py build --stage 2` – builds the stage2 compiler + - `./x.py build --stage 2` – builds the stage2 compiler, along with `std` and + `rustdoc` (which doesn't take too long) - Running tests (see the [section on running tests](../tests/running.html) for more details): - `./x.py test library/std` – runs the `#[test]` tests from `std` diff --git a/src/conventions.md b/src/conventions.md index 582de0364..0b0a951c3 100644 --- a/src/conventions.md +++ b/src/conventions.md @@ -8,17 +8,25 @@ chapter covers [formatting](#formatting), [coding for correctness](#cc), # Formatting and the tidy script rustc is moving towards the [Rust standard coding style][fmt]. -This is enforced by the "tidy" script and can be mostly -automated using `./x.py fmt`. -As the output of [rustfmt] is not completely stable, -formatting this repository using `cargo fmt` is not recommended. +However, for now we don't use stable `rustfmt`; we use a pinned version with a +special config, so this may result in different style from normal [`rustfmt`]. +Therefore, formatting this repository using `cargo fmt` is not recommended. -The tidy script runs automatically when you do `./x.py test` and can be run -in isolation with `./x.py test tidy`. +Instead, formatting should be done using `./x.py fmt`. It's a good habit to run +`./x.py fmt` before every commit, as this reduces conflicts later. + +Formatting is checked by the "tidy" script. It runs automatically when you do +`./x.py test` and can be run in isolation with `./x.py test tidy`. `./x.py fmt +--check` also works. + +If you want to use format-on-save in your editor, the pinned version of +`rustfmt` is built under `build//stage0/bin/rustfmt`. You'll have to +pass the `--edition=2021` argument yourself when calling +`rustfmt` directly. [fmt]: https://github.com/rust-dev-tools/fmt-rfcs -[rustfmt]:https://github.com/rust-lang/rustfmt +[`rustfmt`]:https://github.com/rust-lang/rustfmt diff --git a/src/getting-started.md b/src/getting-started.md index 40e68cf0e..2b780ac4b 100644 --- a/src/getting-started.md +++ b/src/getting-started.md @@ -96,174 +96,28 @@ See [this chapter][config] for more info about configuration. [config]: ./building/how-to-build-and-run.md#create-a-configtoml -### Building and Testing `rustc` +### Common `x.py` commands -Here is a summary of the different commands for reference, but you probably -should still read the rest of the section: +Here are the basic invocations of the `x.py` commands most commonly used when +working on `rustc`, `std`, `rustdoc`, and other tools. | Command | When to use it | | --- | --- | -| `./x.py check` | Quick check to see if things compile; [rust-analyzer can run this automatically for you][rust-analyzer] | -| `./x.py build --stage 0 [library/std]` | Build only the standard library, without building the compiler | -| `./x.py build library/std` | Build just the 1st stage of the compiler, along with the standard library; this is faster than building stage 2 and usually good enough | -| `./x.py build --keep-stage 1 library/std` | Build the 1st stage of the compiler and skips rebuilding the standard library; this is useful after you've done an ordinary stage1 build to skip compilation time, but it can cause weird problems. (Just do a regular build to resolve.) | -| `./x.py test [--keep-stage 1]` | Run the test suite using the stage1 compiler | -| `./x.py test --bless [--keep-stage 1]` | Run the test suite using the stage1 compiler _and_ update expected test output. | -| `./x.py build --stage 2 compiler/rustc` | Do a full 2-stage build. You almost never want to do this. | -| `./x.py test --stage 2` | Do a full 2-stage build and run all tests. You almost never want to do this. | - -To do a full 2-stage build of the whole compiler, you should run this (after -updating `config.toml` as mentioned above): +| `./x.py check` | Quick check to see if most things compile; [rust-analyzer can run this automatically for you][rust-analyzer] | +| `./x.py build` | Builds `rustc`, `std`, and `rustdoc` | +| `./x.py test` | Runs all tests | +| `./x.py fmt` | Formats all code | -```sh -./x.py build --stage 2 compiler/rustc -``` - -In the process, this will also necessarily build the standard libraries, and it -will build `rustdoc` (which doesn't take too long). - -To build and test everything: - -```sh -./x.py test -``` - -For most contributions, you only need to build stage 1, which saves a lot of time: - -```sh -# Build the compiler (stage 1) -./x.py build library/std - -# Subsequent builds -./x.py build --keep-stage 1 library/std -``` - -This will take a while, especially the first time. Be wary of accidentally -touching or formatting the compiler, as `x.py` will try to recompile it. - -**NOTE**: The `--keep-stage 1` will _assume_ that the stage 0 standard library -does not need to be rebuilt, which is usually true, which will save some time. -However, if you are changing certain parts of the compiler, this may lead to -weird errors. Feel free to ask on [zulip][z] if you are running into issues. - -This runs a ton of tests and takes a long time to complete. If you are -working on `rustc`, you can usually get by with only the [UI tests][uitests]. These -test are mostly for the frontend of the compiler, so if you are working on LLVM -or codegen, this shortcut will _not_ test your changes. You can read more about the -different test suites [in this chapter][testing]. +As written, these commands are reasonable starting points. However, there are +additional options and arguments for each of them that are worth learning for +serious development work. In particular, `./x.py build` and `./x.py test` +provide many ways to compile or test a subset of the code, which can save a lot +of time. [rust-analyzer]: ./building/suggested.html#configuring-rust-analyzer-for-rustc -[uitests]: ./tests/adding.html#ui -[testing]: https://rustc-dev-guide.rust-lang.org/tests/intro.html - -```sh -# First build -./x.py test src/test/ui - -# Subsequent builds -./x.py test src/test/ui --keep-stage 1 -``` - -If your changes impact test output, you can use `--bless` to automatically -update the `.stderr` files of the affected tests: - -```sh -./x.py test src/test/ui --keep-stage 1 --bless -``` - -While working on the compiler, it can be helpful to see if the code just -compiles (similar to `cargo check`) without actually building it. You can do -this with: - -```sh -./x.py check -``` - -This command is really fast (relative to the other commands). It usually -completes in a couple of minutes on my laptop. **A common workflow when working -on the compiler is to make changes and repeatedly check with `./x.py check`. -Then, run the tests as shown above when you think things should work.** - -Finally, the CI ensures that the codebase is using consistent style. To format -the code: - -```sh -# Actually format -./x.py fmt -# Just check formatting, exit with error -./x.py fmt --check -``` - -*Note*: we don't use stable `rustfmt`; we use a pinned version with a special -config, so this may result in different style from normal `rustfmt` if you have -format-on-save turned on. It's a good habit to run `./x.py fmt` before every -commit, as this reduces conflicts later. The pinned version is built under -`build//stage0/bin/rustfmt`, so if you want, you can use it for a -single file or for format-on-save in your editor, which can be faster than `./x.py fmt`. -You'll have to pass the `--edition=2021` argument -yourself when calling `rustfmt` directly. - -One last thing: you can use `RUSTC_LOG=XXX` to get debug logging. [Read more -here][logging]. Notice the `C` in `RUSTC_LOG`. Other than that, it uses normal -[`env_logger`][envlog] or `tracing` syntax. - -[envlog]: https://crates.io/crates/env_logger -[logging]: ./tracing.md - -### Building and Testing `std`/`core`/`alloc`/`test`/`proc_macro`/etc. - -As before, technically the proper way to build one of these libraries is to use -the stage-2 compiler, which of course requires a 2-stage build, described above -(`./x.py build`). - -In practice, though, you don't need to build the compiler unless you are -planning to use a recently added nightly feature. Instead, you can just build -stage 0, which uses the current beta compiler. - -```sh -./x.py build --stage 0 -``` - -```sh -./x.py test --stage 0 library/std -``` - -(The same works for `library/alloc`, `library/core`, etc.) - -### Building and Testing `rustdoc` - -`rustdoc` uses `rustc` internals (and, of course, the standard library), so you -will have to build the compiler and `std` once before you can build `rustdoc`. -As before, you can use `./x.py build` to do this. The first time you build, -the stage-1 compiler will also be built. - -```sh -# First build -./x.py build - -# Subsequent builds -./x.py build --keep-stage 1 -``` - -As with the compiler, you can do a fast check build: - -```sh -./x.py check -``` - -Rustdoc has two types of tests: content tests and UI tests. - -```sh -# Content tests -./x.py test src/test/rustdoc - -# UI tests -./x.py test src/test/rustdoc-ui - -# Both at once -./x.py test src/test/rustdoc src/test/rustdoc-ui -``` +See the chapters on [building](./building/how-to-build-and-run.md) and +[testing](./tests/intro.md) for more details. ### Contributing code to other Rust projects diff --git a/src/rustdoc.md b/src/rustdoc.md index 66cb496dc..6c30a556c 100644 --- a/src/rustdoc.md +++ b/src/rustdoc.md @@ -1,6 +1,9 @@ # Rustdoc overview -Rustdoc actually uses the rustc internals directly. It lives in-tree with the +`rustdoc` uses `rustc` internals (and, of course, the standard library), so you +will have to build the compiler and `std` once before you can build `rustdoc`. + +`rustdoc` lives in-tree with the compiler and standard library. This chapter is about how it works. For information about Rustdoc's features and how to use them, see the [Rustdoc book](https://doc.rust-lang.org/nightly/rustdoc/). @@ -39,9 +42,11 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) * Run `./x.py setup tools` before getting started. This will configure `x.py` with nice settings for developing rustdoc and other tools, including downloading a copy of rustc rather than building it. +* Use `./x.py check` to quickly check for compile errors. * Use `./x.py build` to make a usable rustdoc you can run on other projects. * Add `library/test` to be able to use `rustdoc --test`. + * Add `--keep-stage 1` on subsequent runs to avoid rebuilding some things. * Run `rustup toolchain link stage2 build/$TARGET/stage2` to add a custom toolchain called `stage2` to your rustup environment. After running that, `cargo +stage2 doc` in any directory will build with @@ -52,7 +57,8 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) * If you want to copy those docs to a webserver, copy all of `build/$TARGET/doc`, since that's where the CSS, JS, fonts, and landing page are. -* Use `./x.py test src/test/rustdoc*` to run the tests using a stage1 rustdoc. +* Use `./x.py test src/test/rustdoc*` to run the tests using a stage1 + rustdoc. * See [Rustdoc internals] for more information about tests. ## Code structure diff --git a/src/tests/running.md b/src/tests/running.md index c91a27bf2..a204fcab0 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -36,7 +36,11 @@ modifying rustc to see if things are generally working correctly would be the following: ```bash +# First build ./x.py test src/test/ui + +# Subsequent builds (optional, but can save time) +./x.py test src/test/ui --keep-stage 1 ``` This will run the `ui` test suite. Of course, the choice @@ -92,6 +96,13 @@ tests for components you did not change at all. build; therefore, while the tests **usually** work fine with stage 1, there are some limitations. +### Run all tests using a stage 2 compiler + +```bash +./x.py test --stage 2 +``` +You almost never need to do this. + ## Run unit tests on the compiler/library You may want to run unit tests on a specific file with following: From c7adafb2b752fa1e544958e2cca7c93f91b426bb Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Thu, 6 Jan 2022 11:39:57 +1100 Subject: [PATCH 2/3] Address review comments in #1286. --- src/conventions.md | 5 ++--- src/getting-started.md | 4 ++-- src/rustdoc.md | 1 - src/tests/running.md | 4 ---- 4 files changed, 4 insertions(+), 10 deletions(-) diff --git a/src/conventions.md b/src/conventions.md index 0b0a951c3..16828c496 100644 --- a/src/conventions.md +++ b/src/conventions.md @@ -16,9 +16,8 @@ Therefore, formatting this repository using `cargo fmt` is not recommended. Instead, formatting should be done using `./x.py fmt`. It's a good habit to run `./x.py fmt` before every commit, as this reduces conflicts later. -Formatting is checked by the "tidy" script. It runs automatically when you do -`./x.py test` and can be run in isolation with `./x.py test tidy`. `./x.py fmt ---check` also works. +Formatting is checked by the `tidy` script. It runs automatically when you do +`./x.py test` and can be run in isolation with `./x.py fmt --check`. If you want to use format-on-save in your editor, the pinned version of `rustfmt` is built under `build//stage0/bin/rustfmt`. You'll have to diff --git a/src/getting-started.md b/src/getting-started.md index 2b780ac4b..a5e67456e 100644 --- a/src/getting-started.md +++ b/src/getting-started.md @@ -116,8 +116,8 @@ of time. [rust-analyzer]: ./building/suggested.html#configuring-rust-analyzer-for-rustc -See the chapters on [building](./building/how-to-build-and-run.md) and -[testing](./tests/intro.md) for more details. +See the chapters on [building](./building/how-to-build-and-run.md), +[testing](./tests/intro.md), and [rustdoc](./rustdoc.md) for more details. ### Contributing code to other Rust projects diff --git a/src/rustdoc.md b/src/rustdoc.md index 6c30a556c..d6063de73 100644 --- a/src/rustdoc.md +++ b/src/rustdoc.md @@ -46,7 +46,6 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) * Use `./x.py build` to make a usable rustdoc you can run on other projects. * Add `library/test` to be able to use `rustdoc --test`. - * Add `--keep-stage 1` on subsequent runs to avoid rebuilding some things. * Run `rustup toolchain link stage2 build/$TARGET/stage2` to add a custom toolchain called `stage2` to your rustup environment. After running that, `cargo +stage2 doc` in any directory will build with diff --git a/src/tests/running.md b/src/tests/running.md index a204fcab0..ba87c5f9a 100644 --- a/src/tests/running.md +++ b/src/tests/running.md @@ -36,11 +36,7 @@ modifying rustc to see if things are generally working correctly would be the following: ```bash -# First build ./x.py test src/test/ui - -# Subsequent builds (optional, but can save time) -./x.py test src/test/ui --keep-stage 1 ``` This will run the `ui` test suite. Of course, the choice From 10dc74f4be0781e908a5123e3184672c2bb36616 Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Thu, 6 Jan 2022 12:52:38 +1100 Subject: [PATCH 3/3] Address more review comments in #1286. --- src/building/how-to-build-and-run.md | 2 +- src/rustdoc.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index 339662780..cd4891358 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -182,7 +182,7 @@ You almost never need to do this. If you are working on the standard library, you probably don't need to build the compiler unless you are planning to use a recently added nightly feature. -Instead, you can just build stage 0, which uses the current beta compiler. +Instead, you can just build using the bootstrap compiler. ```bash ./x.py build --stage 0 library/std diff --git a/src/rustdoc.md b/src/rustdoc.md index d6063de73..7d07c0414 100644 --- a/src/rustdoc.md +++ b/src/rustdoc.md @@ -42,7 +42,7 @@ does is call the `main()` that's in this crate's `lib.rs`, though.) * Run `./x.py setup tools` before getting started. This will configure `x.py` with nice settings for developing rustdoc and other tools, including downloading a copy of rustc rather than building it. -* Use `./x.py check` to quickly check for compile errors. +* Use `./x.py check src/tools/rustdoc` to quickly check for compile errors. * Use `./x.py build` to make a usable rustdoc you can run on other projects. * Add `library/test` to be able to use `rustdoc --test`.