Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify "how to build and run" section #1528

Merged
merged 5 commits into from
Jan 1, 2023
Merged

Simplify "how to build and run" section #1528

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
791d829
Remove initial section on submodules
jyn514 Dec 8, 2022
a74e8c0
Remove unnecessary detail in building chapter
jyn514 Dec 8, 2022
9d08a7d
Move "create a config.toml" after the section explaning x.py
jyn514 Dec 8, 2022
3e01ef1
Move information about dependencies to the rust-lang/rust readme
jyn514 Dec 8, 2022
50ee97b
Update sentence about LLVM to match the new defaults
jyn514 Dec 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 27 additions & 93 deletions src/building/how-to-build-and-run.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The compiler is built using a tool called `x.py`. You will need to
have Python installed to run it.

For instructions on how to install Python and other prerequisites,
see [the next page](./prerequisites.md).
see [the `rust-lang/rust` README][readme].

## Get the source code

Expand All @@ -13,6 +13,7 @@ the standard library (including `core`, `alloc`, `test`, `proc_macro`, etc),
and a bunch of tools (e.g. `rustdoc`, the bootstrapping infrastructure, etc).

[repo]: https://github.com/rust-lang/rust
[readme]: https://github.com/rust-lang/rust#building-on-a-unix-like-system

The very first step to work on `rustc` is to clone the repository:

Expand All @@ -21,67 +22,15 @@ git clone https://github.com/rust-lang/rust.git
cd rust
```

There are also submodules for things like LLVM, `clippy`, `miri`, etc. The
build tool will automatically clone and sync these for you. But if you want to,
you can do the following:

```sh
# first time
git submodule update --init --recursive

# subsequent times (to pull new commits)
git submodule update
```

## Create a `config.toml`

To start, run `./x.py setup`. This will do some initialization and create a
`config.toml` for you with reasonable defaults. These defaults are specified
indirectly via the `profile` setting, which points to one of the TOML files in
`src/bootstrap/defaults.`

Alternatively, you can write `config.toml` by hand. See `config.toml.example`
for all the available settings and explanations of them. The following settings
are of particular interest, and `config.toml.example` has full explanations.

You may want to change some of the following settings (and possibly others, such as
`llvm.ccache`):

```toml
[llvm]
# Whether to use Rust CI built LLVM instead of locally building it.
download-ci-llvm = true # Download a pre-built LLVM?
assertions = true # LLVM assertions on?
ccache = "/path/to/ccache" # Use ccache when building LLVM?

[rust]
debug-logging = true # Leave debug! and trace! calls in rustc?
incremental = true # Build rustc with incremental compilation?
```

If you set `download-ci-llvm = true`, in some circumstances, such as when
updating the version of LLVM used by `rustc`, you may want to temporarily
disable this feature. See the ["Updating LLVM" section] for more.

["Updating LLVM" section]: ../backend/updating-llvm.md#feature-updates

If you have already built `rustc` and you change settings related to LLVM, then you may have to
execute `rm -rf build` for subsequent configuration changes to take effect. Note that `./x.py
clean` will not cause a rebuild of LLVM.

## What is `x.py`?

`x.py` is the script used to orchestrate the tooling in the `rustc` repository.
It is the script that can build docs, run tests, and compile `rustc`.
It is the now preferred way to build `rustc` and it replaces the old makefiles
from before. Below are the different ways to utilize `x.py` in order to
effectively deal with the repo for various common tasks.
`x.py` is the build tool for the `rust` repository. It can build docs, run tests, and compile the
compiler and standard library.

This chapter focuses on the basics to be productive, but
if you want to learn more about `x.py`, read its README.md
[here](https://github.com/rust-lang/rust/blob/master/src/bootstrap/README.md).
To read more about the bootstrap process and why `x.py` is necessary,
[read this chapter][bootstrap].
if you want to learn more about `x.py`, [read this chapter][bootstrap].

[bootstrap]: ./bootstrapping.md

### Running `x.py` slightly more conveniently

Expand All @@ -91,48 +40,26 @@ of a checkout. It also looks up the appropriate version of `python` to use.

You can install it with `cargo install --path src/tools/x`.

[bootstrap]: ./bootstrapping.md

## Building the Compiler
## Create a `config.toml`

To build a compiler, run `./x.py build`. This will build up to the stage1 compiler,
including `rustdoc`, producing a usable compiler toolchain from the source
code you have checked out.
To start, run `./x.py setup`. This will do some initialization and create a
`config.toml` for you with reasonable defaults.

Note that building will require a relatively large amount of storage space.
You may want to have upwards of 10 or 15 gigabytes available to build the compiler.
Alternatively, you can write `config.toml` by hand. See `config.toml.example` for all the available
settings and explanations of them. See `src/bootstrap/defaults` for common settings to change.

There are many flags you can pass to the build command of `x.py` that can be
beneficial to cutting down compile times or fitting other things you might
need to change. They are:

```txt
Options:
-v, --verbose use verbose output (-vv for very verbose)
-i, --incremental use incremental compilation
--config FILE TOML configuration file for build
--build BUILD build target of the stage0 compiler
--host HOST host targets to build
--target TARGET target targets to build
--on-fail CMD command to run on failure
--stage N stage to build
--keep-stage N stage to keep without recompiling
--src DIR path to the root of the Rust checkout
-j, --jobs JOBS number of jobs to run in parallel
-h, --help print this help message
```
If you have already built `rustc` and you change settings related to LLVM, then you may have to
execute `rm -rf build` for subsequent configuration changes to take effect. Note that `./x.py
clean` will not cause a rebuild of LLVM.

For hacking, often building the stage 1 compiler is enough, which saves a lot
of time. But for final testing and release, the stage 2 compiler is used.
## Building the Compiler

`./x.py check` is really fast to build the Rust compiler.
It is, in particular, very useful when you're doing some kind of
"type-based refactoring", like renaming a method, or changing the
signature of some function.
Note that building will require a relatively large amount of storage space.
You may want to have upwards of 10 or 15 gigabytes available to build the compiler.

Once you've created a `config.toml`, you are now ready to run
`x.py`. There are a lot of options here, but let's start with what is
probably the best "go to" command for building a local rust:
probably the best "go to" command for building a local compiler:

```bash
./x.py build library
Expand All @@ -156,6 +83,10 @@ see [the section on avoiding rebuilds for std][keep-stage].

[keep-stage]: ./suggested.md#faster-builds-with---keep-stage

Sometimes you don't need a full build. When doing some kind of
"type-based refactoring", like renaming a method, or changing the
signature of some function, you can use `./x.py check` instead for a much faster build.

Note that this whole command just gives you a subset of the full `rustc`
build. The **full** `rustc` build (what you get with `./x.py build
--stage 2 compiler/rustc`) has quite a few more steps:
Expand All @@ -177,6 +108,9 @@ Instead, you can just build using the bootstrap compiler.
./x.py build --stage 0 library
```

If you choose the `library` profile when running `x.py setup`, you can omit `--stage 0` (it's the
default).

## Creating a rustup toolchain

Once you have successfully built `rustc`, you will have created a bunch
Expand Down Expand Up @@ -285,7 +219,7 @@ in other sections:
- `./x.py build` – builds everything using the stage 1 compiler,
not just up to `std`
- `./x.py build --stage 2` – builds everything with the stage 2 compiler including
`rustdoc` (which doesn't take too long)
`rustdoc`
- Running tests (see the [section on running tests](../tests/running.html) for
more details):
- `./x.py test library/std` – runs the unit tests and integration tests from `std`
Expand Down
54 changes: 3 additions & 51 deletions src/building/prerequisites.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,48 +2,7 @@

## Dependencies

Before building the compiler, you need the following things installed:

* `python` 3 or 2.7 (under the name `python`; `python2` or `python3` will not work)
* `curl`
* `git`
* `ssl` which comes in `libssl-dev` or `openssl-devel`
* `pkg-config` if you are compiling on Linux and targeting Linux
* `libstdc++-static` may be required on some Linux distributions such as Fedora and Ubuntu

If building LLVM from source (the default), you'll need additional tools:

* `g++`, `clang++`, or MSVC with versions listed on <!-- date-check: Aug 2022 -->
[LLVM's documentation](https://releases.llvm.org/13.0.0/docs/GettingStarted.html#host-c-toolchain-both-compiler-and-standard-library)
* `ninja`, or GNU `make` 3.81 or later (ninja is recommended, especially on Windows)
* `cmake` 3.13.4 or later

Otherwise, you'll need LLVM installed and `llvm-config` in your path.
See [this section for more info][sysllvm].

[sysllvm]: ./new-target.md#using-pre-built-llvm

### Windows

* Install [winget](https://github.com/microsoft/winget-cli)

`winget` is a Windows package manager. It will make package installation easy
on Windows.

Run the following in a terminal:

```powershell
winget install -e Python.Python.3
winget install -e Kitware.CMake
```

If any of those is installed already, winget will detect it. Then edit your system's `PATH` variable
and add: `C:\Program Files\CMake\bin`. See
[this guide on editing the system `PATH`](https://www.java.com/en/download/help/path.html) from the
Java documentation.

For more information about building on Windows,
see [the `rust-lang/rust` README](https://github.com/rust-lang/rust#building-on-windows).
See [the `rust-lang/rust` README](https://github.com/rust-lang/rust#dependencies).

## Hardware

Expand All @@ -65,8 +24,8 @@ powerful, a common strategy is to only use `./x.py check` on your local machine
and let the CI build test your changes when you push to a PR branch.

Building the compiler takes more than half an hour on my moderately powerful
laptop. The first time you build the compiler, LLVM will also be built unless
you use CI-built LLVM ([see here][config]).
laptop. We suggest downloading LLVM from CI so you don't have to build it from source
([see here][config]).

Like `cargo`, the build system will use as many cores as possible. Sometimes
this can cause you to run low on memory. You can use `-j` to adjust the number
Expand All @@ -80,10 +39,3 @@ longer (especially after a rebase), but will save a ton of space from the
incremental caches.

[config]: ./how-to-build-and-run.md#create-a-configtoml

## `rustc` and toolchain installation

Follow the installation given in the [Rust book][install] to install a working
`rustc` and the necessary C/++ toolchain on your platform.

[install]: https://doc.rust-lang.org/book/ch01-01-installation.html