@@ -4,105 +4,31 @@ This is an in-progress README which is targeted at helping to explain how Rust
4
4
is bootstrapped and in general, some of the technical details of the build
5
5
system.
6
6
7
- ## Using rustbuild
7
+ Note that this README only covers internal information, not how to use the tool.
8
+ Please check [ bootstrapping dev guide] [ bootstrapping-dev-guide ] for further information.
8
9
9
- The rustbuild build system has a primary entry point, a top level ` x.py ` script:
10
+ [ bootstrapping-dev-guide ] : https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html
10
11
11
- ``` sh
12
- $ python ./x.py build
13
- ```
14
-
15
- Note that if you're on Unix, you should be able to execute the script directly:
16
-
17
- ``` sh
18
- $ ./x.py build
19
- ```
20
-
21
- The script accepts commands, flags, and arguments to determine what to do:
22
-
23
- * ` build ` - a general purpose command for compiling code. Alone, ` build ` will
24
- bootstrap the entire compiler, and otherwise, arguments passed indicate what to
25
- build. For example:
26
-
27
- ```
28
- # build the whole compiler
29
- ./x.py build --stage 2
30
-
31
- # build the stage1 compiler
32
- ./x.py build
33
-
34
- # build stage0 libstd
35
- ./x.py build --stage 0 library/std
36
-
37
- # build a particular crate in stage0
38
- ./x.py build --stage 0 library/test
39
- ```
40
-
41
- If files that would normally be rebuilt from stage 0 are dirty, the rebuild can be
42
- overridden using ` --keep-stage 0 ` . Using ` --keep-stage n ` will skip all steps
43
- that belong to stage n or earlier:
44
-
45
- ```
46
- # build stage 1, keeping old build products for stage 0
47
- ./x.py build --keep-stage 0
48
- ```
49
-
50
- * ` test ` - a command for executing unit tests. Like the ` build ` command, this
51
- will execute the entire test suite by default, and otherwise, it can be used to
52
- select which test suite is run:
53
-
54
- ```
55
- # run all unit tests
56
- ./x.py test
57
-
58
- # execute tool tests
59
- ./x.py test tidy
60
-
61
- # execute the UI test suite
62
- ./x.py test tests/ui
63
-
64
- # execute only some tests in the UI test suite
65
- ./x.py test tests/ui --test-args substring-of-test-name
66
-
67
- # execute tests in the standard library in stage0
68
- ./x.py test --stage 0 library/std
69
-
70
- # execute tests in the core and standard library in stage0,
71
- # without running doc tests (thus avoid depending on building the compiler)
72
- ./x.py test --stage 0 --no-doc library/core library/std
12
+ ## Introduction
73
13
74
- # execute all doc tests
75
- ./x.py test src/doc
76
- ```
14
+ The build system defers most of the complicated logic managing invocations
15
+ of rustc and rustdoc to Cargo itself. However, moving through various stages
16
+ and copying artifacts is still necessary for it to do. Each time rustbuild
17
+ is invoked, it will iterate through the list of predefined steps and execute
18
+ each serially in turn if it matches the paths passed or is a default rule.
19
+ For each step rustbuild relies on the step internally being incremental and
20
+ parallel. Note, though, that the ` -j ` parameter to rustbuild gets forwarded
21
+ to appropriate test harnesses and such.
77
22
78
- * ` doc ` - a command for building documentation. Like above, can take arguments
79
- for what to document.
80
-
81
- ## Configuring rustbuild
82
-
83
- rustbuild offers a TOML-based configuration system with a ` config.toml `
84
- file. An example of this configuration can be found at ` config.toml.example ` ,
85
- and the configuration file can also be passed as ` --config path/to/config.toml `
86
- if the build system is being invoked manually (via the python script).
87
-
88
- You can generate a config.toml using ` ./configure ` options if you want to automate creating the file without having to edit it.
89
-
90
- Finally, rustbuild makes use of the [ cc-rs crate] which has [ its own
91
- method] [ env-vars ] of configuring C compilers and C flags via environment
92
- variables.
93
-
94
- [ cc-rs crate ] : https://github.com/alexcrichton/cc-rs
95
- [ env-vars ] : https://github.com/alexcrichton/cc-rs#external-configuration-via-environment-variables
96
-
97
- ## Build stages
23
+ ## Build phases
98
24
99
25
The rustbuild build system goes through a few phases to actually build the
100
26
compiler. What actually happens when you invoke rustbuild is:
101
27
102
- 1 . The entry point script, ` x.py ` is run. This script is
103
- responsible for downloading the stage0 compiler/Cargo binaries, and it then
104
- compiles the build system itself (this folder). Finally, it then invokes the
105
- actual ` bootstrap ` binary build system.
28
+ 1 . The entry point script( ` x ` for unix like systems , ` x.ps1 ` for windows systems,
29
+ ` x.py ` cross-platform) is run. This script is responsible for downloading the stage0
30
+ compiler/Cargo binaries, and it then compiles the build system itself (this folder).
31
+ Finally, it then invokes the actual ` bootstrap ` binary build system.
106
32
2 . In Rust, ` bootstrap ` will slurp up all configuration, perform a number of
107
33
sanity checks (whether compilers exist, for example), and then start building the
108
34
stage0 artifacts.
@@ -115,24 +41,6 @@ compiler. What actually happens when you invoke rustbuild is:
115
41
The goal of each stage is to (a) leverage Cargo as much as possible and failing
116
42
that (b) leverage Rust as much as possible!
117
43
118
- ## Incremental builds
119
-
120
- You can configure rustbuild to use incremental compilation with the
121
- ` --incremental ` flag:
122
-
123
- ``` sh
124
- $ ./x.py build --incremental
125
- ```
126
-
127
- The ` --incremental ` flag will store incremental compilation artifacts
128
- in ` build/<host>/stage0-incremental ` . Note that we only use incremental
129
- compilation for the stage0 -> stage1 compilation -- this is because
130
- the stage1 compiler is changing, and we don't try to cache and reuse
131
- incremental artifacts across different versions of the compiler.
132
-
133
- You can always drop the ` --incremental ` to build as normal (but you
134
- will still be using the local nightly as your bootstrap).
135
-
136
44
## Directory Layout
137
45
138
46
This build system houses all output under the ` build ` directory, which looks
@@ -236,63 +144,31 @@ build/
236
144
# system will link (using hard links) output from stageN-{std,rustc} into
237
145
# each of these directories.
238
146
#
239
- # In theory, there is no extra build output in these directories.
147
+ # In theory these are working rustc sysroot directories, meaning there is
148
+ # no extra build output in these directories.
240
149
stage1/
241
150
stage2/
242
151
stage3/
243
152
```
244
153
245
- ## Cargo projects
246
-
247
- The current build is unfortunately not quite as simple as ` cargo build ` in a
248
- directory, but rather the compiler is split into three different Cargo projects:
249
-
250
- * ` library/std ` - the standard library
251
- * ` library/test ` - testing support, depends on libstd
252
- * ` compiler/rustc ` - the actual compiler itself
253
-
254
- Each "project" has a corresponding Cargo.lock file with all dependencies, and
255
- this means that building the compiler involves running Cargo three times. The
256
- structure here serves two goals:
257
-
258
- 1 . Facilitating dependencies coming from crates.io. These dependencies don't
259
- depend on ` std ` , so libstd is a separate project compiled ahead of time
260
- before the actual compiler builds.
261
- 2 . Splitting "host artifacts" from "target artifacts". That is, when building
262
- code for an arbitrary target, you don't need the entire compiler, but you'll
263
- end up needing libraries like libtest that depend on std but also want to use
264
- crates.io dependencies. Hence, libtest is split out as its own project that
265
- is sequenced after ` std ` but before ` rustc ` . This project is built for all
266
- targets.
267
-
268
- There is some loss in build parallelism here because libtest can be compiled in
269
- parallel with a number of rustc artifacts, but in theory, the loss isn't too bad!
270
-
271
- ## Build tools
272
-
273
- We've actually got quite a few tools that we use in the compiler's build system
274
- and for testing. To organize these, each tool is a project in ` src/tools ` with a
275
- corresponding ` Cargo.toml ` . All tools are compiled with Cargo (currently having
276
- independent ` Cargo.lock ` files) and do not currently explicitly depend on the
277
- compiler or standard library. Compiling each tool is sequenced after the
278
- appropriate libstd/libtest/librustc compile above.
279
-
280
154
## Extending rustbuild
281
155
282
- So, you'd like to add a feature to the rustbuild build system or just fix a bug.
283
- Great! One of the major motivational factors for moving away from ` make ` is that
284
- Rust is in theory much easier to read, modify, and write. If you find anything
285
- excessively confusing, please open an issue on this, and we'll try to get it
286
- documented or simplified, pronto.
156
+ When you use the bootstrap system, you'll call it through the entry point script
157
+ (` x ` , ` x.ps1 ` , or ` x.py ` ). However, most of the code lives in ` src/bootstrap ` .
158
+ ` bootstrap ` has a difficult problem: it is written in Rust, but yet it is run
159
+ before the Rust compiler is built! To work around this, there are two components
160
+ of bootstrap: the main one written in rust, and ` bootstrap.py ` . ` bootstrap.py `
161
+ is what gets run by entry point script. It takes care of downloading the ` stage0 `
162
+ compiler, which will then build the bootstrap binary written in Rust.
287
163
288
- First up, you'll probably want to read over the documentation above, as that'll
289
- give you a high level overview of what rustbuild is doing. You also probably
290
- want to play around a bit yourself by just getting it up and running before you
291
- dive too much into the actual build system itself.
164
+ Because there are two separate codebases behind ` x.py ` , they need to
165
+ be kept in sync. In particular, both ` bootstrap.py ` and the bootstrap binary
166
+ parse ` config.toml ` and read the same command line arguments. ` bootstrap.py `
167
+ keeps these in sync by setting various environment variables, and the
168
+ programs sometimes have to add arguments that are explicitly ignored, to be
169
+ read by the other.
292
170
293
- After that, each module in rustbuild should have enough documentation to keep
294
- you up and running. Some general areas that you may be interested in modifying
295
- are:
171
+ Some general areas that you may be interested in modifying are:
296
172
297
173
* Adding a new build tool? Take a look at ` bootstrap/tool.rs ` for examples of
298
174
other tools.
@@ -320,8 +196,9 @@ A 'major change' includes
320
196
Changes that do not affect contributors to the compiler or users
321
197
building rustc from source don't need an update to ` VERSION ` .
322
198
323
- If you have any questions, feel free to reach out on the ` #t-infra ` channel in
324
- the [ Rust Zulip server] [ rust-zulip ] or ask on internals.rust-lang.org. When
325
- you encounter bugs, please file issues on the rust-lang/rust issue tracker.
199
+ If you have any questions, feel free to reach out on the ` #t-infra/bootstrap ` channel
200
+ at [ Rust Bootstrap Zulip server] [ rust-bootstrap- zulip ] . When you encounter bugs,
201
+ please file issues on the [ Rust issue tracker] [ rust-issue-tracker ] .
326
202
327
- [ rust-zulip ] : https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra
203
+ [ rust-bootstrap-zulip ] : https://rust-lang.zulipchat.com/#narrow/stream/t-infra.2Fbootstrap
204
+ [ rust-issue-tracker ] : https://github.com/rust-lang/rust/issues
0 commit comments