diff --git a/src/how-to-build-and-run.md b/src/how-to-build-and-run.md
index 3ad9a52ae..66ba8efda 100644
--- a/src/how-to-build-and-run.md
+++ b/src/how-to-build-and-run.md
@@ -49,28 +49,36 @@ use-jemalloc = false
### Running x.py and building a stage1 compiler
-One thing to keep in mind is that `rustc` is a _bootstrapping_ compiler. That
-is, since `rustc` is written in Rust, we need to use an older version of the
-compiler to compile the newer version. In particular, the newer version of the
-compiler, `libstd`, and other tooling may use some unstable features
-internally. The result is the compiling `rustc` is done in stages.
-
-- **Stage 0:** the stage0 compiler can be your existing
- (perhaps older version of)
- Rust compiler, the current _beta_ compiler or you may download the binary
- from the internet.
-- **Stage 1:** the code in your clone (for new version)
- is then compiled with the stage0
- compiler to produce the stage1 compiler.
- However, it was built with an older compiler (stage0),
- so to optimize the stage1 compiler we go to next stage.
-- **Stage 2:** we rebuild our stage1 compiler with itself
- to produce the stage2 compiler (i.e. it builds
- itself) to have all the _latest optimizations_.
-- _(Optional)_ **Stage 3**: to sanity check of our new compiler,
- we can build it again
- with stage2 compiler which must be identical to itself,
- unless something has broken.
+One thing to keep in mind is that `rustc` is a _bootstrapping_
+compiler. That is, since `rustc` is written in Rust, we need to use an
+older version of the compiler to compile the newer version. In
+particular, the newer version of the compiler, `libstd`, and other
+tooling may use some unstable features internally. The result is that
+compiling `rustc` is done in stages:
+
+- **Stage 0:** the stage0 compiler is usually the current _beta_ compiler
+ (`x.py` will download it for you); you can configure `x.py` to use something
+ else, though.
+- **Stage 1:** the code in your clone (for new version) is then
+ compiled with the stage0 compiler to produce the stage1 compiler.
+ However, it was built with an older compiler (stage0), so to
+ optimize the stage1 compiler we go to next stage.
+ - (In theory, the stage1 compiler is functionally identical to the
+ stage2 compiler, but in practice there are subtle differences. In
+ particular, the stage1 compiler itself was built by stage0 and
+ hence not by the source in your working directory: this means that
+ the symbol names used in the compiler source may not match the
+ symbol names that would have been made by the stage1 compiler.
+ This can be important when using dynamic linking (e.g., with
+ derives. Sometimes this means that some tests don't work when run
+ with stage1.)
+- **Stage 2:** we rebuild our stage1 compiler with itself to produce
+ the stage2 compiler (i.e. it builds itself) to have all the _latest
+ optimizations_. (By default, we copy the stage1 libraries for use by
+ the stage2 compiler, since they ought to be identical.)
+- _(Optional)_ **Stage 3**: to sanity check of our new compiler, we
+ can build the libraries with the stage2 compiler. The result ought
+ to be identical to before, unless something has broken.
For hacking, often building the stage 1 compiler is enough, but for
final testing and release, the stage 2 compiler is used.
@@ -80,6 +88,8 @@ It is, in particular, very useful when you're doing some kind of
"type-based refactoring", like renaming a method, or changing the
signature of some function.
+
+
Once you've created a config.toml, you are now ready to run
`x.py`. There are a lot of options here, but let's start with what is
probably the best "go to" command for building a local rust:
@@ -88,27 +98,39 @@ probably the best "go to" command for building a local rust:
> ./x.py build -i --stage 1 src/libstd
```
-What this command will do is the following:
+This may *look* like it only builds libstd, but that is not the case.
+What this command does is the following:
+
+- Build libstd using the stage0 compiler (using incremental)
+- Build librustc using the stage0 compiler (using incremental)
+ - This produces the stage1 compiler
+- Build libstd using the stage1 compiler (cannot use incremental)
+
+This final product (stage1 compiler + libs built using that compiler)
+is what you need to build other rust programs.
-- Using the beta compiler (also called stage 0), it will build the
- standard library and rustc from the `src` directory. The resulting
- compiler is called the "stage 1" compiler.
- - During this build, the `-i` (or `--incremental`) switch enables incremental
- compilation, so that if you later rebuild after editing things in
- `src`, you can save a bit of time.
-- Using this stage 1 compiler, it will build the standard library.
- (this is what the `src/libstd`) means.
+Note that the command includes the `-i` switch. This enables incremental
+compilation. This will be used to speed up the first two steps of the process:
+in particular, if you make a small change, we ought to be able to use your old
+results to make producing the stage1 **compiler** faster.
-This is just a subset of the full rustc build. The **full** rustc build
-(what you get if you just say `./x.py build`) has quite a few more steps:
+Unfortunately, incremental cannot be used to speed up making the
+stage1 libraries. This is because incremental only works when you run
+the *same compiler* twice in a row. In this case, we are building a
+*new stage1 compiler* every time. Therefore, the old incremental
+results may not apply. **As a result, you will probably find that
+building the stage1 libstd is a bottleneck for you** -- but fear not,
+there is a (hacky) workaround. See [the section on "recommended
+workflows"](#workflow) below.
-- Build stage1 rustc with stage0 compiler.
-- Build libstd with stage1 compiler (up to here is the same).
-- Build rustc from `src` again, this time with the stage1 compiler
- (this part is new).
+Note that this whole command just gives you a subset of the full rustc
+build. The **full** rustc build (what you get if you just say `./x.py
+build`) has quite a few more steps:
+
+- Build librustc and rustc with the stage1 compiler.
- The resulting compiler here is called the "stage2" compiler.
- Build libstd with stage2 compiler.
-- Build librustdoc and a bunch of other things.
+- Build librustdoc and a bunch of other things with the stage2 compiler.
@@ -148,6 +170,66 @@ release: 1.25.0-dev
LLVM version: 4.0
```
+
+
+### Suggested workflows for faster builds of the compiler
+
+There are two workflows that are useful for faster builders of the
+compiler.
+
+**Check, check, and check again.** The first workflow, which is useful
+when doing simple refactorings, is to run `./x.py check`
+continuously. Here you are just checking that the compiler can
+**build**, but often that is all you need (e.g., when renaming a
+method). You can then run `./x.py build` when you actually need to
+run tests.
+
+In fact, it is sometimes useful to put off tests even when you are not
+100% sure the code will work. You can then keep building up
+refactoring commits and only run the tests at some later time. You can
+then use `git bisect` to track down **precisely** which commit caused
+the problem. A nice side-effect of this style is that you are left
+with a fairly fine-grained set of commits at the end, all of which
+build and pass tests. This often helps reviewing.
+
+**Incremental builds with `--keep-stage`.** Sometimes just checking
+whether the compiler builds is not enough. A common example is that
+you need to add a `debug!` statement to inspect the value of some
+state or better understand the problem. In that case, you really need
+a full build. By leveraging incremental, though, you can often get
+these builds to complete very fast (e.g., around 30 seconds): the only
+catch is this requires a bit of fudging and may produce compilers that
+don't work (but that is easily detected and fixed).
+
+The sequence of commands you want is as follows:
+
+- Initial build: `./x.py build -i --stage 1 src/libstd`
+ - As [documented above](#command), this will build a functional
+ stage1 compiler
+- Subsequent builds: `./x.py build -i --stage 1 src/libstd --keep-stage 1`
+ - Note that we added the `--keep-stage 1` flag here
+
+The effect of `--keep-stage 1` is that we just *assume* that the old
+standard library can be re-used. If you are editing the compiler, this
+is almost always true: you haven't changed the standard library, after
+all. But sometimes, it's not true: for example, if you are editing
+the "metadata" part of the compiler, which controls how the compiler
+encodes types and other states into the `rlib` files, or if you are
+editing things that wind up in the metadata (such as the definition of
+the MIR).
+
+**The TL;DR is that you might get weird behavior from a compile when
+using `--keep-stage 1`** -- for example, strange
+[ICEs](appendix/glossary.html) or other panics. In that case, you
+should simply remove the `--keep-stage 1` from the command and
+rebuild. That ought to fix the problem.
+
+You can also use `--keep-stage 1` when running tests. Something like
+this:
+
+- Initial test run: `./x.py test -i --stage 1 src/test/ui`
+- Subsequent test run: `./x.py test -i --stage 1 src/test/ui --keep-stage 1`
+
### Other x.py commands
Here are a few other useful x.py commands. We'll cover some of them in detail
diff --git a/src/tests/running.md b/src/tests/running.md
index fdc603a54..d410b79c3 100644
--- a/src/tests/running.md
+++ b/src/tests/running.md
@@ -61,21 +61,24 @@ filtering for tests that include "issue-1234" in the name.
## Using incremental compilation
-You can further enable the `--incremental` flag to save additional time in subsequent rebuilds:
+You can further enable the `--incremental` flag to save additional
+time in subsequent rebuilds:
```bash
> ./x.py test --stage 1 src/test/ui --incremental --test-args issue-1234
```
-If you don't want to include the flag with every command, you can enable it in the `config.toml`, too:
+If you don't want to include the flag with every command, you can
+enable it in the `config.toml`, too:
```toml
# Whether to always use incremental compilation when building rustc
incremental = true
```
-Note that incremental compilation will use more disk space than usual. If disk space is a
-concern for you, you might want to check the size of the `build` directory from time to time.
+Note that incremental compilation will use more disk space than
+usual. If disk space is a concern for you, you might want to check the
+size of the `build` directory from time to time.
## Running tests manually