Skip to content

Commit

Permalink
Streamline "Getting Started" (rust-lang#1279)
Browse files Browse the repository at this point in the history 
* Move `x.py` intro section before first use, and shorten it.

* Improve `x.py setup` docs.

In "Getting Started", strip it back to the bare minimum. Some of this is
moved into the later section.

In the later section, add notable details like config.toml.example how
and `profile` works. Also make the config.toml example more concise.

* Move details about the repository.

Less detail in "Getting Started", more in the later sections.

* Move details about the prereqs.

Less detail in "Getting Started", more in the later sections.
  • Loading branch information
@nnethercote
nnethercote authored Dec 24, 2021
1 parent 1349a37 commit 2c2ebc2
Show file tree
Hide file tree
Showing 3 changed files with 89 additions and 138 deletions.
56 changes: 38 additions & 18 deletions src/building/how-to-build-and-run.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,43 +11,63 @@ see [the next page](./prerequisites.md).

## Get the source code

The main repository is [`rust-lang/rust`][repo]. This contains the compiler,
the standard library (including `core`, `alloc`, `test`, `proc_macro`, etc),
and a bunch of tools (e.g. `rustdoc`, the bootstrapping infrastructure, etc).

[repo]: https://github.com/rust-lang/rust

The very first step to work on `rustc` is to clone the repository:

```bash
git clone https://github.com/rust-lang/rust.git
cd rust
```

There are also submodules for things like LLVM, `clippy`, `miri`, etc. The
build tool will automatically clone and sync these for you. But if you want to,
you can do the following:

```sh
# first time
git submodule update --init --recursive

# subsequent times (to pull new commits)
git submodule update
```

## Create a `config.toml`

To start, run `./x.py setup`. This will create a `config.toml` with reasonable defaults.
To start, run `./x.py setup`. This will do some initialization and create a
`config.toml` for you with reasonable defaults. These defaults are specified
indirectly via the `profile` setting, which points to one of the TOML files in
`src/bootstrap/defaults.`

Alternatively, you can write `config.toml` by hand. See `config.toml.example`
for all the available settings and explanations of them. The following settings
are of particular interest, and `config.toml.example` has full explanations.

You may also want to change some of the following settings (and possibly others, such as
You may want to change some of the following settings (and possibly others, such as
`llvm.ccache`):

```toml
[llvm]
# Whether to use Rust CI built LLVM instead of locally building it.
download-ci-llvm = true

# Indicates whether the LLVM assertions are enabled or not
assertions = true
download-ci-llvm = true # Download a pre-built LLVM?
assertions = true # LLVM assertions on?
ccache = "/path/to/ccache" # Use ccache when building LLVM?

[rust]
# Whether or not to leave debug! and trace! calls in the rust binary.
# Overrides the `debug-assertions` option, if defined.
#
# Defaults to rust.debug-assertions value
#
# If you see a message from `tracing` saying
# `max_level_info` is enabled and means logging won't be shown,
# set this value to `true`.
debug-logging = true

# Whether to always use incremental compilation when building rustc
incremental = true
debug-logging = true # Leave debug! and trace! calls in rustc?
incremental = true # Build rustc with incremental compilation?
```

If you set `download-ci-llvm = true`, in some circumstances, such as when
updating the version of LLVM used by `rustc`, you may want to temporarily
disable this feature. See the ["Updating LLVM" section] for more.

["Updating LLVM" section]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html?highlight=download-ci-llvm#feature-updates

If you have already built `rustc` and you change settings related to LLVM, then you may have to
execute `rm -rf build` for subsequent configuration changes to take effect. Note that `./x.py
clean` will not cause a rebuild of LLVM.
Expand Down
35 changes: 29 additions & 6 deletions src/building/prerequisites.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,17 +43,40 @@ see [the `rust-lang/rust` README](https://github.com/rust-lang/rust#building-on-

## Hardware

These are not so much requirements as _recommendations_:

* ~15GB of free disk space (~25GB or more if doing incremental builds).
* \>= 8GB RAM
* \>= 2 cores
* Internet access
You will need an internet connection to build. The bootstrapping process
involves updating git submodules and downloading a beta compiler. It doesn't
need to be super fast, but that can help.

There are no strict hardware requirements, but building the compiler is
computationally expensive, so a beefier machine will help, and I wouldn't
recommend trying to build on a Raspberry Pi! We recommend the following.
* 30GB+ of free disk space. Otherwise, you will have to keep
clearing incremental caches. More space is better, the compiler is a bit of a
hog; it's a problem we are aware of.
* 8GB+ RAM
* 2+ cores. Having more cores really helps. 10 or 20 or more is not too many!

Beefier machines will lead to much faster builds. If your machine is not very
powerful, a common strategy is to only use `./x.py check` on your local machine
and let the CI build test your changes when you push to a PR branch.

Building the compiler takes more than half an hour on my moderately powerful
laptop. The first time you build the compiler, LLVM will also be built unless
you use CI-built LLVM ([see here][config]).

Like `cargo`, the build system will use as many cores as possible. Sometimes
this can cause you to run low on memory. You can use `-j` to adjust the number
concurrent jobs. If a full build takes more than ~45 minutes to an hour, you
are probably spending most of the time swapping memory in and out; try using
`-j1`.

If you don't have too much free disk space, you may want to turn off
incremental compilation ([see here][config]). This will make compilation take
longer (especially after a rebase), but will save a ton of space from the
incremental caches.

[config]: ./how-to-build-and-run.md#create-a-configtoml

## `rustc` and toolchain installation

Follow the installation given in the [Rust book][install] to install a working
Expand Down
136 changes: 22 additions & 114 deletions src/getting-started.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,149 +44,57 @@ just create noise, so we ask that you be mindful of the fact that the

## Cloning and Building

The main repository is [`rust-lang/rust`][repo]. This contains the compiler,
the standard library (including `core`, `alloc`, `test`, `proc_macro`, etc),
and a bunch of tools (e.g. `rustdoc`, the bootstrapping infrastructure, etc).

[repo]: https://github.com/rust-lang/rust

There are also a bunch of submodules for things like LLVM, `clippy`, `miri`,
etc. You don't need to clone these immediately, but the build tool will
automatically clone and sync them (more on this later).

[**Take a look at the "Suggested Workflows" chapter for some helpful
advice.**][suggested]

[suggested]: ./building/suggested.md

### System Requirements

[**See this chapter for detailed software requirements.**](./building/prerequisites.md)
Most notably, you will need Python 2 or 3 to run `x.py`.

There are no hard hardware requirements, but building the compiler is
computationally expensive, so a beefier machine will help, and I wouldn't
recommend trying to build on a Raspberry Pi :P
Internet access is required.

- Recommended >=30GB of free disk space; otherwise, you will have to keep
clearing incremental caches. More space is better, the compiler is a bit of a
hog; it's a problem we are aware of.
- Recommended >=8GB RAM.
- Recommended >=2 cores; having more cores really helps.
- You will need an internet connection to build; the bootstrapping process
involves updating git submodules and downloading a beta compiler. It doesn't
need to be super fast, but that can help.
The most notable software requirement is that you will need Python 2 or 3, but
there are various others.

Building the compiler takes more than half an hour on my moderately powerful
laptop. The first time you build the compiler, LLVM will also be built unless
you use CI-built LLVM ([see below][configsec]).
The following hardware is recommended.
* 30GB+ of free disk space.
* 8GB+ RAM
* 2+ cores

[configsec]: #configuring-the-compiler
More powerful machines will lead to much faster builds. There are various
strategies to work around lesser hardware in the following chapters.

Like `cargo`, the build system will use as many cores as possible. Sometimes
this can cause you to run low on memory. You can use `-j` to adjust the number
concurrent jobs. If a full build takes more than ~45 minutes to an hour,
you are probably spending most of the time swapping memory in and out;
try using `-j1`.
See [this chapter][prereqs] for more details about software and hardware prerequisites.

On a slow machine, the build times for rustc are very painful. Consider using
`./x.py check` instead of a full build and letting the automated tests run
when you push to GitHub.

If you don't have too much free disk space, you may want to turn off
incremental compilation ([see below][configsec]). This will make
compilation take longer (especially after a rebase),
but will save a ton of space from the incremental caches.
[prereqs]: ./building/prerequisites.md

### Cloning

You can just do a normal git clone:

```sh
git clone https://github.com/rust-lang/rust.git
cd rust
```

You don't need to clone the submodules at this time. But if you want to, you
can do the following:
### `x.py` Intro

```sh
# first time
git submodule update --init --recursive
`rustc` is a [bootstrapping] compiler, which makes it more complex than a
typical Rust program. As a result, you cannot use Cargo to build it. Instead
you must use the special tool `x.py`. It is used for the things Cargo is
normally used for: building, testing, creating releases, formatting, etc.

# subsequent times (to pull new commits)
git submodule update
```
[bootstrapping]: ./building/bootstrapping.md

### Configuring the Compiler

The compiler has a configuration file which contains a ton of settings. We will
provide some recommendations here that should work for most, but [check out
this chapter for more info][config].

[config]: ./building/how-to-build-and-run.md#create-a-configtoml

In the top level of the repo:

```sh
$ ./x.py setup
```

This will walk you through an interactive setup for `x.py` that looks like this:

```
$ ./x.py setup
Welcome to the Rust project! What do you want to do with x.py?
a) Contribute to the standard library
b) Contribute to the compiler
c) Contribute to the compiler, and also modify LLVM or codegen
d) Install Rust from source
Please choose one (a/b/c/d): a
`x.py` will now use the configuration at /home/joshua/rustc2/src/bootstrap/defaults/config.toml.library
To get started, try one of the following commands:
- `x.py check`
- `x.py build`
- `x.py test library/std`
- `x.py doc`
For more suggestions, see https://rustc-dev-guide.rust-lang.org/building/suggested.html
```

Note that by default, `./x.py setup` will use CI-built LLVM if available for your
platform so that you don't need to build LLVM in addition to building the
compiler. In some circumstances, such as when updating the version of LLVM used
by `rustc`, you may want to temporarily disable this feature. See the ["Updating
LLVM" section] for more.
This will do some initialization and walk you through an interactive setup to
create `config.toml`, the primary configuration file.

If you want to download LLVM from CI without running `./x.py setup`, you can set
the `download-ci-llvm` option to `true` in your `config.toml`:
See [this chapter][config] for more info about configuration.

```toml
[llvm]
download-ci-llvm = true
```

["Updating LLVM" section]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html?highlight=download-ci-llvm#feature-updates

### `x.py` Intro

`rustc` is a _bootstrapping_ compiler, which means that it is written in Rust
and thus needs to be compiled by itself. So where do you
get the original compiler from? We use the current beta compiler
to build a new compiler. Then, we use that compiler to build itself. Thus,
`rustc` has a 2-stage build. You can read more about bootstrapping
[here][boot], but you don't need to know much more to contribute.

[boot]: ./building/bootstrapping.md

We have a special tool `x.py` that drives this process. It is used for
building the compiler, the standard libraries, and `rustdoc`. It is also used
for driving CI and building the final release artifacts.

Unfortunately, a proper 2-stage build takes a long time depending on your
hardware, but it is the only correct way to build everything (e.g. it's what
the CI and release processes use). **However, in most cases, you can get by
without a full 2-stage build**. In the following section, we give instructions
for how to do "the correct thing", but then we also give various tips to speed
things up.
[config]: ./building/how-to-build-and-run.md#create-a-configtoml

### Building and Testing `rustc`

Expand Down

0 comments on commit 2c2ebc2

Please sign in to comment.