Skip to content

Commit

Permalink
Auto merge of #62549 - ehuss:update-cargo-vendor, r=alexcrichton
Browse files Browse the repository at this point in the history 
Update cargo-vendor usage

This contains a variety of updates to clean up the usage of cargo-vendor.

- Remove the install step for the old cargo-vendor now that it is built-in to cargo and available in the stage0 install.
- Update installation instructions, dealing with vendoring. The current instructions of running `sudo ./x.py install` is broken, it will almost always fail (since the vendor directory doesn't exist). Since the steps for properly handling this are numerous, I'm recommending removing the suggestion to use `sudo` altogether.
- If the sudo-forced-vendoring detects that the vendor directory is not available, abort with instructions on how to fix.
- Now that cargo-vendor is built-in, automatically run it if it looks like it is needed.
- Update instructions on how to install cargo.
- Remove the unused markdown link references in README/CONTRIBUTING. This reverts most of #44935. These references don't do anything if they are unused.

Closes #49269
cc #61142 #48771 #40108
  • Loading branch information
@bors
bors committed Jul 11, 2019
2 parents 4bb6b4a + 06c3256 commit 4da4deb
Show file tree
Hide file tree
Showing 4 changed files with 71 additions and 95 deletions.
9 changes: 1 addition & 8 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,6 @@ Speaking of tests, Rust has a comprehensive test suite. More information about
it can be found [here][rctd].

### External Dependencies
[external-dependencies]: #external-dependencies

Currently building Rust will also build the following external projects:

Expand Down Expand Up @@ -209,7 +208,6 @@ Breakage is not allowed in the beta and stable channels, and must be addressed
before the PR is merged.

#### Breaking Tools Built With The Compiler
[breaking-tools-built-with-the-compiler]: #breaking-tools-built-with-the-compiler

Rust's build system builds a number of tools that make use of the
internals of the compiler. This includes
Expand Down Expand Up @@ -242,7 +240,7 @@ Here are those same steps in detail:
`config.toml.example` in the root directory of the Rust repository.
Set `submodules = false` in the `[build]` section. This will prevent `x.py`
from resetting to the original branch after you make your changes. If you
need to [update any submodules to their latest versions][updating-submodules],
need to [update any submodules to their latest versions](#updating-submodules),
see the section of this file about that for more information.
2. (optional) Run `./x.py test src/tools/rustfmt` (substituting the submodule
that broke for `rustfmt`). Fix any errors in the submodule (and possibly others).
Expand All @@ -256,7 +254,6 @@ Here are those same steps in detail:
8. (optional) Send a PR to rust-lang/rust updating the submodule.

#### Updating submodules
[updating-submodules]: #updating-submodules

These instructions are specific to updating `rustfmt`, however they may apply
to the other submodules as well. Please help by improving these instructions
Expand Down Expand Up @@ -310,7 +307,6 @@ This should change the version listed in `Cargo.lock` to the new version you upd
the submodule to. Running `./x.py build` should work now.

## Writing Documentation
[writing-documentation]: #writing-documentation

Documentation improvements are very welcome. The source of `doc.rust-lang.org`
is located in `src/doc` in the tree, and standard API documentation is generated
Expand All @@ -337,7 +333,6 @@ tracker in that repo is also a great way to find things that need doing. There
are issues for beginners and advanced compiler devs alike!

## Issue Triage
[issue-triage]: #issue-triage

Sometimes, an issue will stay open, even though the bug has been fixed. And
sometimes, the original bug may go stale because something has changed in the
Expand Down Expand Up @@ -405,7 +400,6 @@ If you're looking for somewhere to start, check out the [E-easy][eeasy] tag.
[rfcbot]: https://github.com/anp/rfcbot-rs/

## Out-of-tree Contributions
[out-of-tree-contributions]: #out-of-tree-contributions

There are a number of other ways to contribute to Rust that don't deal with
this repository.
Expand All @@ -425,7 +419,6 @@ valuable!
[community-library]: https://github.com/rust-lang/rfcs/labels/A-community-library

## Helpful Links and Information
[helpful-info]: #helpful-info

For people new to Rust, and just starting to contribute, or even for
more seasoned developers, some useful places to look for information
Expand Down
68 changes: 27 additions & 41 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,22 @@ standard library, and documentation.
[Rust]: https://www.rust-lang.org

## Quick Start
[quick-start]: #quick-start

Read ["Installation"] from [The Book].

["Installation"]: https://doc.rust-lang.org/book/ch01-01-installation.html
[The Book]: https://doc.rust-lang.org/book/index.html

## Installing from Source
[building-from-source]: #building-from-source

_Note: If you wish to contribute to the compiler, you should read
[this chapter](https://rust-lang.github.io/rustc-guide/how-to-build-and-run.html)
of the rustc-guide instead of this section._
_Note: If you wish to contribute to the compiler, you should read [this
chapter][rustcguidebuild] of the rustc-guide instead of this section._

The Rust build system has a Python script called `x.py` to bootstrap building
the compiler. More information about it may be found by running `./x.py --help`
or reading the [rustc guide][rustcguidebuild].

[rustcguidebuild]: https://rust-lang.github.io/rustc-guide/how-to-build-and-run.html

### Building on *nix
1. Make sure you have installed the dependencies:
Expand All @@ -39,43 +42,36 @@ of the rustc-guide instead of this section._

[source]: https://github.com/rust-lang/rust

3. Build and install:

```sh
$ ./x.py build && sudo ./x.py install
```
3. Configure the build settings:

If after running `sudo ./x.py install` you see an error message like
The Rust build system uses a file named `config.toml` in the root of the
source tree to determine various configuration settings for the build.
Copy the default `config.toml.example` to `config.toml` to get started.

```
error: failed to load source for a dependency on 'cc'
```sh
$ cp config.toml.example config.toml
```

then run these two commands and then try `sudo ./x.py install` again:
It is recommended that if you plan to use the Rust build system to create
an installation (using `./x.py install`) that you set the `prefix` value
in the `[install]` section to a directory that you have write permissions.

```
$ cargo install cargo-vendor
```
4. Build and install:

```sh
$ ./x.py build && ./x.py install
```
$ cargo vendor
```

> ***Note:*** Install locations can be adjusted by copying the config file
> from `./config.toml.example` to `./config.toml`, and
> adjusting the `prefix` option under `[install]`. Various other options, such
> as enabling debug information, are also supported, and are documented in
> the config file.

When complete, `sudo ./x.py install` will place several programs into
`/usr/local/bin`: `rustc`, the Rust compiler, and `rustdoc`, the
When complete, `./x.py install` will place several programs into
`$PREFIX/bin`: `rustc`, the Rust compiler, and `rustdoc`, the
API-documentation tool. This install does not include [Cargo],
Rust's package manager, which you may also want to build.
Rust's package manager. To build and install Cargo, you may
run `./x.py install cargo` or set the `build.extended` key in
`config.toml` to `true` to build and install all tools.
[Cargo]: https://github.com/rust-lang/cargo
### Building on Windows
[building-on-windows]: #building-on-windows
There are two prominent ABIs in use on Windows: the native (MSVC) ABI used by
Visual Studio, and the GNU ABI used by the GCC toolchain. Which version of Rust
Expand All @@ -85,7 +81,6 @@ for interop with GNU software built using the MinGW/MSYS2 toolchain use the GNU
build.
#### MinGW
[windows-mingw]: #windows-mingw
[MSYS2][msys2] can be used to easily build Rust on Windows:
Expand Down Expand Up @@ -126,7 +121,6 @@ build.
```
#### MSVC
[windows-msvc]: #windows-msvc
MSVC builds of Rust additionally require an installation of Visual Studio 2017
(or later) so `rustc` can use its linker. The simplest way is to get the
Expand Down Expand Up @@ -155,7 +149,6 @@ by manually calling the appropriate vcvars file before running the bootstrap.
```
#### Specifying an ABI
[specifying-an-abi]: #specifying-an-abi
Each specific ABI can also be used from either environment (for example, using
the GNU ABI in PowerShell) by using an explicit build triple. The available
Expand All @@ -169,11 +162,10 @@ Windows build triples are:
The build triple can be specified by either specifying `--build=<triple>` when
invoking `x.py` commands, or by copying the `config.toml` file (as described
in Building From Source), and modifying the `build` option under the `[build]`
section.
in [Installing From Source](#installing-from-source)), and modifying the
`build` option under the `[build]` section.
### Configure and Make
[configure-and-make]: #configure-and-make
While it's not the recommended build system, this project also provides a
configure script and makefile (the latter of which just invokes `x.py`).
Expand All @@ -188,7 +180,6 @@ When using the configure script, the generated `config.mk` file may override the
`config.mk` file.
## Building Documentation
[building-documentation]: #building-documentation
If you’d like to build the documentation, it’s almost the same:
Expand All @@ -201,7 +192,6 @@ the ABI used. I.e., if the ABI was `x86_64-pc-windows-msvc`, the directory will
`build\x86_64-pc-windows-msvc\doc`.
## Notes
[notes]: #notes
Since the Rust compiler is written in Rust, it must be built by a
precompiled "snapshot" version of itself (made in an earlier stage of
Expand All @@ -224,7 +214,6 @@ There is more advice about hacking on Rust in [CONTRIBUTING.md].
[CONTRIBUTING.md]: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md
## Getting Help
[getting-help]: #getting-help
The Rust community congregates in a few places:
Expand All @@ -237,7 +226,6 @@ The Rust community congregates in a few places:
[users.rust-lang.org]: https://users.rust-lang.org/
## Contributing
[contributing]: #contributing
To contribute to Rust, please see [CONTRIBUTING](CONTRIBUTING.md).
Expand All @@ -258,7 +246,6 @@ Also, you may find the [rustdocs for the compiler itself][rustdocs] useful.
[rustdocs]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc/
## License
[license]: #license
Rust is primarily distributed under the terms of both the MIT license
and the Apache License (Version 2.0), with portions covered by various
Expand All @@ -268,7 +255,6 @@ See [LICENSE-APACHE](LICENSE-APACHE), [LICENSE-MIT](LICENSE-MIT), and
[COPYRIGHT](COPYRIGHT) for details.
## Trademark
[trademark]: #trademark
The Rust programming language is an open source, community project governed
by a core team. It is also sponsored by the Mozilla Foundation (“Mozilla”),
Expand Down
67 changes: 43 additions & 24 deletions src/bootstrap/bootstrap.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -735,6 +735,47 @@ def set_dev_environment(self):
"""Set download URL for development environment"""
self._download_url = 'https://dev-static.rust-lang.org'

def check_vendored_status(self):
"""Check that vendoring is configured properly"""
vendor_dir = os.path.join(self.rust_root, 'vendor')
if 'SUDO_USER' in os.environ and not self.use_vendored_sources:
if os.environ.get('USER') != os.environ['SUDO_USER']:
self.use_vendored_sources = True
print('info: looks like you are running this command under `sudo`')
print(' and so in order to preserve your $HOME this will now')
print(' use vendored sources by default.')
if not os.path.exists(vendor_dir):
print('error: vendoring required, but vendor directory does not exist.')
print(' Run `cargo vendor` without sudo to initialize the '
'vendor directory.')
raise Exception("{} not found".format(vendor_dir))

if self.use_vendored_sources:
if not os.path.exists('.cargo'):
os.makedirs('.cargo')
with output('.cargo/config') as cargo_config:
cargo_config.write(
"[source.crates-io]\n"
"replace-with = 'vendored-sources'\n"
"registry = 'https://example.com'\n"
"\n"
"[source.vendored-sources]\n"
"directory = '{}/vendor'\n"
.format(self.rust_root))
else:
if os.path.exists('.cargo'):
shutil.rmtree('.cargo')

def ensure_vendored(self):
"""Ensure that the vendored sources are available if needed"""
vendor_dir = os.path.join(self.rust_root, 'vendor')
# Note that this does not handle updating the vendored dependencies if
# the rust git repository is updated. Normal development usually does
# not use vendoring, so hopefully this isn't too much of a problem.
if self.use_vendored_sources and not os.path.exists(vendor_dir):
run([self.cargo(), "vendor"],
verbose=self.verbose, cwd=self.rust_root)


def bootstrap(help_triggered):
"""Configure, fetch, build and run the initial bootstrap"""
Expand Down Expand Up @@ -776,30 +817,7 @@ def bootstrap(help_triggered):

build.use_locked_deps = '\nlocked-deps = true' in build.config_toml

if 'SUDO_USER' in os.environ and not build.use_vendored_sources:
if os.environ.get('USER') != os.environ['SUDO_USER']:
build.use_vendored_sources = True
print('info: looks like you are running this command under `sudo`')
print(' and so in order to preserve your $HOME this will now')
print(' use vendored sources by default. Note that if this')
print(' does not work you should run a normal build first')
print(' before running a command like `sudo ./x.py install`')

if build.use_vendored_sources:
if not os.path.exists('.cargo'):
os.makedirs('.cargo')
with output('.cargo/config') as cargo_config:
cargo_config.write("""
[source.crates-io]
replace-with = 'vendored-sources'
registry = 'https://example.com'
[source.vendored-sources]
directory = '{}/vendor'
""".format(build.rust_root))
else:
if os.path.exists('.cargo'):
shutil.rmtree('.cargo')
build.check_vendored_status()

data = stage0_data(build.rust_root)
build.date = data['date']
Expand All @@ -815,6 +833,7 @@ def bootstrap(help_triggered):
build.build = args.build or build.build_triple()
build.download_stage0()
sys.stdout.flush()
build.ensure_vendored()
build.build_bootstrap()
sys.stdout.flush()

Expand Down
22 changes: 0 additions & 22 deletions src/bootstrap/dist.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -935,8 +935,6 @@ impl Step for Src {
}
}

const CARGO_VENDOR_VERSION: &str = "0.1.22";

#[derive(Debug, PartialOrd, Ord, Copy, Clone, Hash, PartialEq, Eq)]
pub struct PlainSourceTarball;

Expand Down Expand Up @@ -998,26 +996,6 @@ impl Step for PlainSourceTarball {

// If we're building from git sources, we need to vendor a complete distribution.
if builder.rust_info.is_git() {
// Get cargo-vendor installed, if it isn't already.
let mut has_cargo_vendor = false;
let mut cmd = Command::new(&builder.initial_cargo);
for line in output(cmd.arg("install").arg("--list")).lines() {
has_cargo_vendor |= line.starts_with("cargo-vendor ");
}
if !has_cargo_vendor {
let mut cmd = builder.cargo(
builder.compiler(0, builder.config.build),
Mode::ToolBootstrap,
builder.config.build,
"install"
);
cmd.arg("--force")
.arg("--debug")
.arg("--vers").arg(CARGO_VENDOR_VERSION)
.arg("cargo-vendor");
builder.run(&mut cmd);
}

// Vendor all Cargo dependencies
let mut cmd = Command::new(&builder.initial_cargo);
cmd.arg("vendor")
Expand Down

0 comments on commit 4da4deb

Please sign in to comment.