2
2
3
3
<!-- toc -->
4
4
5
- Now that we have [ seen what the compiler does] ( ./overview.md ) , let's take a
6
- look at the structure of the [ ` rust-lang/rust ` ] repository, where the rustc
7
- source code lives.
5
+ Now that we have [ seen what the compiler does] [ orgch ] ,
6
+ let's take a look at the structure of the [ ` rust-lang/rust ` ] repository,
7
+ where the rustc source code lives.
8
8
9
9
[ `rust-lang/rust` ] : https://github.com/rust-lang/rust
10
10
11
- > You may find it helpful to read the [ "Overview of the compiler"] ( ./overview.md )
11
+ > You may find it helpful to read the [ "Overview of the compiler"] [ orgch ]
12
12
> chapter, which introduces how the compiler works, before this one.
13
13
14
+ [ orgch ] : ./overview.md
15
+
14
16
## Workspace structure
15
17
16
- The ` rust-lang/rust ` repository consists of a single large cargo workspace
17
- containing the compiler, the standard libraries (` core ` , ` alloc ` , ` std ` ,
18
- ` proc_macro ` , etc), and ` rustdoc ` , along with the build system and a bunch of
19
- tools and submodules for building a full Rust distribution.
18
+ The [ ` rust-lang/rust ` ] repository consists of a single large cargo workspace
19
+ containing the compiler, the standard libraries ([ ` core ` ] , [ ` alloc ` ] , [ ` std ` ] ,
20
+ [ ` proc_macro ` ] , [ ` etc ` ] ), and [ ` rustdoc ` ] , along with the build system and a
21
+ bunch of tools and submodules for building a full Rust distribution.
20
22
21
23
The repository consists of three main directories:
22
24
23
- - ` compiler/ ` contains the source code for ` rustc ` . It consists of many crates
25
+ - [ ` compiler/ ` ] contains the source code for ` rustc ` . It consists of many crates
24
26
that together make up the compiler.
25
27
26
- - ` library/ ` contains the standard libraries (` core ` , ` alloc ` , ` std ` ,
27
- ` proc_macro ` , ` test ` ), as well as the Rust runtime (` backtrace ` , ` rtstartup ` ,
28
- ` lang_start ` ).
28
+ - [ ` library/ ` ] contains the standard libraries ([ ` core ` ] , [ ` alloc ` ] , [ ` std ` ] ,
29
+ [ ` proc_macro ` ] , [ ` test ` ] ), as well as the Rust runtime ([ ` backtrace ` ] , [ ` rtstartup ` ] ,
30
+ [ ` lang_start ` ] ).
29
31
30
- - ` tests/ ` contains the compiler tests.
32
+ - [ ` tests/ ` ] contains the compiler tests.
31
33
32
- - ` src/ ` contains the source code for rustdoc, clippy, cargo, the build system,
34
+ - [ ` src/ ` ] contains the source code for [ ` rustdoc ` ] , [ ` clippy ` ] , [ ` cargo ` ] , the build system,
33
35
language docs, etc.
34
36
37
+ [ `alloc` ] : https://github.com/rust-lang/rust/tree/master/library/alloc
38
+ [ `backtrace` ] : https://github.com/rust-lang/backtrace-rs/
39
+ [ `cargo` ] : https://github.com/rust-lang/cargo
40
+ [ `clippy` ] : https://github.com/rust-lang/rust/tree/master/src/tools/clippy
41
+ [ `compiler/` ] : https://github.com/rust-lang/rust/tree/master/compiler
42
+ [ `core` ] : https://github.com/rust-lang/rust/tree/master/library/core
43
+ [ `etc` ] : https://github.com/rust-lang/rust/tree/master/src/etc
44
+ [ `lang_start` ] : https://github.com/rust-lang/rust/blob/master/library/std/src/rt.rs
45
+ [ `library/` ] : https://github.com/rust-lang/rust/tree/master/library
46
+ [ `proc_macro` ] : https://github.com/rust-lang/rust/tree/master/library/proc_macro
47
+ [ `rtstartup` ] : https://github.com/rust-lang/rust/tree/master/library/rtstartup
48
+ [ `rust-lang/rust` ] : https://github.com/rust-lang/rust
49
+ [ `rustdoc` ] : https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc
50
+ [ `src/` ] : https://github.com/rust-lang/rust/tree/master/src
51
+ [ `std` ] : https://github.com/rust-lang/rust/tree/master/library/std
52
+ [ `test` ] : https://github.com/rust-lang/rust/tree/master/library/test
53
+ [ `tests/` ] : https://github.com/rust-lang/rust/tree/master/tests
54
+
35
55
## Compiler
36
56
37
- The compiler is implemented in the various ` compiler/ ` crates.
38
- The ` compiler/ ` crates all have names starting with ` rustc_* ` . These are a
57
+ The compiler is implemented in the various [ ` compiler/ ` ] crates.
58
+ The [ ` compiler/ ` ] crates all have names starting with ` rustc_* ` . These are a
39
59
collection of around 50 interdependent crates ranging in size from tiny to
40
60
huge. There is also the ` rustc ` crate which is the actual binary (i.e. the
41
61
` main ` function); it doesn't actually do anything besides calling the
42
- ` rustc_driver ` crate, which drives the various parts of compilation in other
62
+ [ ` rustc_driver ` ] crate, which drives the various parts of compilation in other
43
63
crates.
44
64
45
65
The dependency structure of these crates is complex, but roughly it is
@@ -58,16 +78,16 @@ something like this:
58
78
[ ` Span ` ] ), or error reporting: [ ` rustc_data_structures ` ] ,
59
79
[ ` rustc_span ` ] , [ ` rustc_errors ` ] , etc.
60
80
61
- [ main ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/fn.main .html
81
+ [ `rustc_data_structures` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_data_structures/index .html
62
82
[ `rustc_driver` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/index.html
83
+ [ `rustc_errors` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/index.html
63
84
[ `rustc_interface` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_interface/index.html
64
85
[ `rustc_middle` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/index.html
65
- [ `rustc_data_structures` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_data_structures/index.html
66
86
[ `rustc_span` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/index.html
67
87
[ `Span` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_span/struct.Span.html
68
- [ `rustc_errors` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/index .html
88
+ [ main ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_driver/fn.main .html
69
89
70
- You can see the exact dependencies by reading the ` Cargo.toml ` for the various
90
+ You can see the exact dependencies by reading the [ ` Cargo.toml ` ] for the various
71
91
crates, just like a normal Rust crate.
72
92
73
93
One final thing: [ ` src/llvm-project ` ] is a submodule for our fork of LLVM.
@@ -78,117 +98,110 @@ compiler can interface with it.
78
98
Most of this book is about the compiler, so we won't have any further
79
99
explanation of these crates here.
80
100
81
- [ `src/llvm-project` ] : https://github.com/rust-lang/rust/tree/master/src/
82
101
[ `compiler/rustc_llvm` ] : https://github.com/rust-lang/rust/tree/master/compiler/rustc_llvm
102
+ [ `src/llvm-project` ] : https://github.com/rust-lang/rust/tree/master/src/
103
+ [ `Cargo.toml` ] : https://github.com/rust-lang/rust/blob/master/Cargo.toml
83
104
84
105
### Big picture
85
106
86
- The dependency structure is influenced by two main factors:
107
+ The dependency structure of the compiler is influenced by two main factors:
87
108
88
109
1 . Organization. The compiler is a _ huge_ codebase; it would be an impossibly
89
110
large crate. In part, the dependency structure reflects the code structure
90
111
of the compiler.
91
- 2 . Compile time. By breaking the compiler into multiple crates, we can take
112
+ 2 . Compile- time. By breaking the compiler into multiple crates, we can take
92
113
better advantage of incremental/parallel compilation using cargo. In
93
114
particular, we try to have as few dependencies between crates as possible so
94
115
that we don't have to rebuild as many crates if you change one.
95
116
96
117
At the very bottom of the dependency tree are a handful of crates that are used
97
118
by the whole compiler (e.g. [ ` rustc_span ` ] ). The very early parts of the
98
- compilation process (e.g. parsing and the AST) depend on only these.
99
-
100
- After the AST is constructed and other early analysis is done, the compiler's [ query system] [ query ]
101
- gets set up. The query system is set up in a clever way using function
102
- pointers. This allows us to break dependencies between crates, allowing more
103
- parallel compilation.
104
- The query system is defined in [ ` rustc_middle ` ] , so nearly all
105
- subsequent parts of the compiler depend on this crate. It is a really large
106
- crate, leading to long compile times. Some efforts have been made to move stuff
107
- out of it with limited success. Another unfortunate side effect is that sometimes
108
- related functionality gets scattered across different crates. For example,
109
- linting functionality is scattered across earlier parts of the crate,
110
- [ ` rustc_lint ` ] , [ ` rustc_middle ` ] , and other places.
111
-
119
+ compilation process (e.g. [ parsing and the Abstract Syntax Tree (` AST ` )] [ parser ] )
120
+ depend on only these.
121
+
122
+ After the [ ` AST ` ] [ parser ] is constructed and other early analysis is done, the
123
+ compiler's [ query system] [ query ] gets set up. The query system is set up in a
124
+ clever way using function pointers. This allows us to break dependencies
125
+ between crates, allowing more parallel compilation. The query system is defined
126
+ in [ ` rustc_middle ` ] , so nearly all subsequent parts of the compiler depend on
127
+ this crate. It is a really large crate, leading to long compile times. Some
128
+ efforts have been made to move stuff out of it with varying success. Another
129
+ side-effect is that sometimes related functionality gets scattered across
130
+ different crates. For example, linting functionality is found across earlier
131
+ parts of the crate, [ ` rustc_lint ` ] , [ ` rustc_middle ` ] , and other places.
132
+
133
+ Ideally there would be fewer, more cohesive crates, with incremental and
134
+ parallel compilation making sure compile times stay reasonable. However,
135
+ incremental and parallel compilation haven't gotten good enough for that yet,
136
+ so breaking things into separate crates has been our solution so far.
137
+
138
+ At the top of the dependency tree is [ ` rustc_driver ` ] and [ ` rustc_interface ` ]
139
+ which is an unstable wrapper around the query system helping drive various
140
+ stages of compilation. Other consumers of the compiler may use this interface
141
+ in different ways (e.g. [ ` rustdoc ` ] or maybe eventually ` rust-analyzer ` ). The
142
+ [ ` rustc_driver ` ] crate first parses command line arguments and then uses
143
+ [ ` rustc_interface ` ] to drive the compilation to completion.
144
+
145
+ [ parser ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_parse/index.html
112
146
[ `rustc_lint` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint/index.html
113
-
114
- Ideally there would be fewer, more
115
- cohesive crates, with incremental and parallel compilation making sure compile
116
- times stay reasonable. However, our incremental and parallel compilation haven't
117
- gotten good enough for that yet, so breaking things into separate crates has
118
- been our solution so far.
119
-
120
- At the top of the dependency tree are the [ ` rustc_interface ` ] and
121
- [ ` rustc_driver ` ] crates. [ ` rustc_interface ` ] is an unstable wrapper around the
122
- query system that helps to drive the various stages of compilation. Other
123
- consumers of the compiler may use this interface in different ways (e.g.
124
- rustdoc or maybe eventually rust-analyzer). The [ ` rustc_driver ` ] crate first
125
- parses command line arguments and then uses [ ` rustc_interface ` ] to drive the
126
- compilation to completion.
127
-
128
147
[ query ] : ./query.md
129
148
130
- [ orgch ] : ./overview.md
131
-
132
149
## rustdoc
133
150
134
- The bulk of ` rustdoc ` is in [ ` librustdoc ` ] . However, the ` rustdoc ` binary
151
+ The bulk of [ ` rustdoc ` ] is in [ ` librustdoc ` ] . However, the [ ` rustdoc ` ] binary
135
152
itself is [ ` src/tools/rustdoc ` ] , which does nothing except call [ ` rustdoc::main ` ] .
136
153
137
- There is also javascript and CSS for the rustdocs in [ ` src/tools/rustdoc-js ` ]
154
+ There is also ` JavaScript ` and ` CSS ` for the docs in [ ` src/tools/rustdoc-js ` ]
138
155
and [ ` src/tools/rustdoc-themes ` ] .
139
156
140
- You can read more about rustdoc in [ this chapter] [ rustdocch ] .
157
+ You can read more about [ ` rustdoc ` ] in [ this chapter] [ `rustdoc` ] .
141
158
142
159
[ `librustdoc` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustdoc/index.html
143
160
[ `rustdoc::main` ] : https://doc.rust-lang.org/nightly/nightly-rustc/rustdoc/fn.main.html
144
- [ `src/tools/rustdoc` ] : https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc
145
161
[ `src/tools/rustdoc-js` ] : https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc-js
146
162
[ `src/tools/rustdoc-themes` ] : https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc-themes
147
-
148
- [ rustdocch ] : ./rustdoc.md
163
+ [ `src/tools/rustdoc` ] : https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc
164
+ [ `rustdoc` ] : ./rustdoc.md
149
165
150
166
## Tests
151
167
152
168
The test suite for all of the above is in [ ` tests/ ` ] . You can read more
153
169
about the test suite [ in this chapter] [ testsch ] .
154
170
155
- The test harness itself is in [ ` src/tools/compiletest ` ] .
156
-
157
- [ testsch ] : ./tests/intro.md
171
+ The test harness is in [ ` src/tools/compiletest/ ` ] [ `compiletest/` ] .
158
172
159
173
[ `tests/` ] : https://github.com/rust-lang/rust/tree/master/tests
160
- [ `src/tools/compiletest` ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest
174
+ [ testsch ] : ./tests/intro.md
161
175
162
176
## Build System
163
177
164
178
There are a number of tools in the repository just for building the compiler,
165
- standard library, rustdoc, etc, along with testing, building a full Rust
179
+ standard library, [ ` rustdoc ` ] , etc, along with testing, building a full Rust
166
180
distribution, etc.
167
181
168
- One of the primary tools is [ ` src/bootstrap ` ] . You can read more about
182
+ One of the primary tools is [ ` src/bootstrap/ ` ] . You can read more about
169
183
bootstrapping [ in this chapter] [ bootstch ] . The process may also use other tools
170
- from ` src/tools/ ` , such as [ ` tidy ` ] or [ ` compiletest ` ] .
171
-
172
- [ `src/bootstrap` ] : https://github.com/rust-lang/rust/tree/master/src/bootstrap
173
- [ `tidy` ] : https://github.com/rust-lang/rust/tree/master/src/tools/tidy
174
- [ `compiletest` ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest
184
+ from [ ` src/tools/ ` ] , such as [ ` tidy/ ` ] or [ ` compiletest/ ` ] .
175
185
186
+ [ `compiletest/` ] : https://github.com/rust-lang/rust/tree/master/src/tools/compiletest
187
+ [ `src/bootstrap/` ] : https://github.com/rust-lang/rust/tree/master/src/bootstrap
188
+ [ `src/tools/` ] : https://github.com/rust-lang/rust/tree/master/src/tools
189
+ [ `tidy/` ] : https://github.com/rust-lang/rust/tree/master/src/tools/tidy
176
190
[ bootstch ] : ./building/bootstrapping/intro.md
177
191
178
192
## Standard library
179
193
180
- The standard library crates are all in ` library/ ` . They have intuitive names
181
- like ` std ` , ` core ` , ` alloc ` , etc. There is also ` proc_macro ` , ` test ` , and
182
- other runtime libraries.
183
-
184
194
This code is fairly similar to most other Rust crates except that it must be
185
- built in a special way because it can use unstable features.
195
+ built in a special way because it can use unstable ([ ` nightly ` ] ) features.
196
+ The standard library is sometimes referred to as [ ` libstd or the "standard facade" ` ] .
197
+
198
+ [ `libstd or the "standard facade"` ] : https://rust-lang.github.io/rfcs/0040-libstd-facade.html
199
+ [ `nightly` ] : https://doc.rust-lang.org/nightly/nightly-rustc/
186
200
187
201
## Other
188
202
189
203
There are a lot of other things in the ` rust-lang/rust ` repo that are related
190
- to building a full Rust distribution. Most of the time you don't need to worry
191
- about them.
204
+ to building a full Rust distribution. Most of the time you don't need to worry about them.
192
205
193
206
These include:
194
207
- [ ` src/ci ` ] : The CI configuration. This actually quite extensive because we
0 commit comments