document keep-stage1 and also try to clarify

src/how-to-build-and-run.md

@@ -49,28 +49,36 @@ use-jemalloc = false
 
 ### Running x.py and building a stage1 compiler
 
-One thing to keep in mind is that `rustc` is a _bootstrapping_ compiler. That
-is, since `rustc` is written in Rust, we need to use an older version of the
-compiler to compile the newer version. In particular, the newer version of the
-compiler, `libstd`, and other tooling may use some unstable features
-internally. The result is the compiling `rustc` is done in stages.
-
-- **Stage 0:** the stage0 compiler can be your existing
-  (perhaps older version of)
-  Rust compiler, the current _beta_ compiler or you may download the binary
-  from the internet.
-- **Stage 1:** the code in your clone (for new version)
-  is then compiled with the stage0
-  compiler to produce the stage1 compiler.
-  However, it was built with an older compiler (stage0),
-  so to optimize the stage1 compiler we go to next stage.
-- **Stage 2:** we rebuild our stage1 compiler with itself
-  to produce the stage2 compiler (i.e. it builds
-  itself) to have all the _latest optimizations_.
-- _(Optional)_ **Stage 3**: to sanity check of our new compiler,
-  we can build it again
-  with stage2 compiler which must be identical to itself,
-  unless something has broken.
+One thing to keep in mind is that `rustc` is a _bootstrapping_
+compiler. That is, since `rustc` is written in Rust, we need to use an
+older version of the compiler to compile the newer version. In
+particular, the newer version of the compiler, `libstd`, and other
+tooling may use some unstable features internally. The result is that
+compiling `rustc` is done in stages:
+
+- **Stage 0:** the stage0 compiler is usually the current _beta_ compiler
+  (`x.py` will download it for you); you can configure `x.py` to use something
+  else, though.
+- **Stage 1:** the code in your clone (for new version) is then
+  compiled with the stage0 compiler to produce the stage1 compiler.
+  However, it was built with an older compiler (stage0), so to
+  optimize the stage1 compiler we go to next stage.
+  - (In theory, the stage1 compiler is functionally identical to the
+    stage2 compiler, but in practice there are subtle differences. In
+    particular, the stage1 compiler itself was built by stage0 and
+    hence not by the source in your working directory: this means that
+    the symbol names used in the compiler source may not match the
+    symbol names that would have been made by the stage1 compiler.
+    This can be important when using dynamic linking (e.g., with
+    derives. Sometimes this means that some tests don't work when run
+    with stage1.)
+- **Stage 2:** we rebuild our stage1 compiler with itself to produce
+  the stage2 compiler (i.e. it builds itself) to have all the _latest
+  optimizations_. (By default, we copy the stage1 libraries for use by
+  the stage2 compiler, since they ought to be identical.)
+- _(Optional)_ **Stage 3**: to sanity check of our new compiler, we
+  can build the libraries with the stage2 compiler. The result ought
+  to be identical to before, unless something has broken.
 
 For hacking, often building the stage 1 compiler is enough, but for
 final testing and release, the stage 2 compiler is used.
@@ -80,6 +88,8 @@ It is, in particular, very useful when you're doing some kind of
 "type-based refactoring", like renaming a method, or changing the
 signature of some function.
 
+<a name=command></a>
+
 Once you've created a config.toml, you are now ready to run
 `x.py`. There are a lot of options here, but let's start with what is
 probably the best "go to" command for building a local rust:
@@ -88,27 +98,39 @@ probably the best "go to" command for building a local rust:
 > ./x.py build -i --stage 1 src/libstd
 ```
 
-What this command will do is the following:
+This may *look* like it only builds libstd, but that is not the case.
+What this command does is the following:
+
+- Build libstd using the stage0 compiler (using incremental)
+- Build librustc using the stage0 compiler (using incremental)
+  - This produces the stage1 compiler
+- Build libstd using the stage1 compiler (cannot use incremental)
+
+This final product (stage1 compiler + libs built using that compiler)
+is what you need to build other rust programs.
 
-- Using the beta compiler (also called stage 0), it will build the
-  standard library and rustc from the `src` directory. The resulting
-  compiler is called the "stage 1" compiler.
-  - During this build, the `-i` (or `--incremental`) switch enables incremental
-    compilation, so that if you later rebuild after editing things in
-    `src`, you can save a bit of time.
-- Using this stage 1 compiler, it will build the standard library.
-  (this is what the `src/libstd`) means.
+Note that the command includes the `-i` switch. This enables incremental
+compilation. This will be used to speed up the first two steps of the process:
+in particular, if you make a small change, we ought to be able to use your old
+results to make producing the stage1 **compiler** faster.
 
-This is just a subset of the full rustc build. The **full** rustc build
-(what you get if you just say `./x.py build`) has quite a few more steps:
+Unfortunately, incremental cannot be used to speed up making the
+stage1 libraries.  This is because incremental only works when you run
+the *same compiler* twice in a row.  In this case, we are building a
+*new stage1 compiler* every time. Therefore, the old incremental
+results may not apply. **As a result, you will probably find that
+building the stage1 libstd is a bottleneck for you** -- but fear not,
+there is a (hacky) workaround.  See [the section on "recommended
+workflows"](#workflow) below.
 
-- Build stage1 rustc with stage0 compiler.
-- Build libstd with stage1 compiler (up to here is the same).
-- Build rustc from `src` again, this time with the stage1 compiler
-  (this part is new).
+Note that this whole command just gives you a subset of the full rustc
+build. The **full** rustc build (what you get if you just say `./x.py
+build`) has quite a few more steps:
+
+- Build librustc and rustc with the stage1 compiler.
   - The resulting compiler here is called the "stage2" compiler.
 - Build libstd with stage2 compiler.
-- Build librustdoc and a bunch of other things.
+- Build librustdoc and a bunch of other things with the stage2 compiler.
 
 <a name=toolchain></a>
 
@@ -148,6 +170,66 @@ release: 1.25.0-dev
 LLVM version: 4.0
 ```
 
+<a name=workflow></a>
+
+### Suggested workflows for faster builds of the compiler
+
+There are two workflows that are useful for faster builders of the
+compiler.
+
+**Check, check, and check again.** The first workflow, which is useful
+when doing simple refactorings, is to run `./x.py check`
+continuously. Here you are just checking that the compiler can
+**build**, but often that is all you need (e.g., when renaming a
+method). You can then run `./x.py build` when you actually need to
+run tests.
+
+In fact, it is sometimes useful to put off tests even when you are not
+100% sure the code will work. You can then keep building up
+refactoring commits and only run the tests at some later time. You can
+then use `git bisect` to track down **precisely** which commit caused
+the problem. A nice side-effect of this style is that you are left
+with a fairly fine-grained set of commits at the end, all of which
+build and pass tests. This often helps reviewing.
+
+**Incremental builds with `--keep-stage`.** Sometimes just checking
+whether the compiler builds is not enough. A common example is that
+you need to add a `debug!` statement to inspect the value of some
+state or better understand the problem. In that case, you really need
+a full build.  By leveraging incremental, though, you can often get
+these builds to complete very fast (e.g., around 30 seconds): the only
+catch is this requires a bit of fudging and may produce compilers that
+don't work (but that is easily detected and fixed).
+
+The sequence of commands you want is as follows:
+
+- Initial build: `./x.py build -i --stage 1 src/libstd`
+  - As [documented above](#command), this will build a functional
+    stage1 compiler
+- Subsequent builds: `./x.py build -i --stage 1 src/libstd --keep-stage 1`
+  - Note that we added the `--keep-stage 1` flag here
+
+The effect of `--keep-stage 1` is that we just *assume* that the old
+standard library can be re-used. If you are editing the compiler, this
+is almost always true: you haven't changed the standard library, after
+all.  But sometimes, it's not true: for example, if you are editing
+the "metadata" part of the compiler, which controls how the compiler
+encodes types and other states into the `rlib` files, or if you are
+editing things that wind up in the metadata (such as the definition of
+the MIR).
+
+**The TL;DR is that you might get weird behavior from a compile when
+using `--keep-stage 1`** -- for example, strange
+[ICEs](appendix/glossary.html) or other panics. In that case, you
+should simply remove the `--keep-stage 1` from the command and
+rebuild.  That ought to fix the problem.
+
+You can also use `--keep-stage 1` when running tests. Something like
+this:
+
+- Initial test run: `./x.py test -i --stage 1 src/test/ui`
+- Subsequent test run: `./x.py test -i --stage 1 src/test/ui --keep-stage 1`
+
 ### Other x.py commands
 
 Here are a few other useful x.py commands. We'll cover some of them in detail

src/tests/running.md

@@ -61,21 +61,24 @@ filtering for tests that include "issue-1234" in the name.
 
 ## Using incremental compilation
 
-You can further enable the `--incremental` flag to save additional time in subsequent rebuilds:
+You can further enable the `--incremental` flag to save additional
+time in subsequent rebuilds:
 
 ```bash
 > ./x.py test --stage 1 src/test/ui --incremental --test-args issue-1234
 ```
 
-If you don't want to include the flag with every command, you can enable it in the `config.toml`, too:
+If you don't want to include the flag with every command, you can
+enable it in the `config.toml`, too:
 
 ```toml
 # Whether to always use incremental compilation when building rustc
 incremental = true
 ```
 
-Note that incremental compilation will use more disk space than usual. If disk space is a
-concern for you, you might want to check the size of the `build` directory from time to time.
+Note that incremental compilation will use more disk space than
+usual. If disk space is a concern for you, you might want to check the
+size of the `build` directory from time to time.
 
 ## Running tests manually