Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x.py: run rustup toolchain link in setup #89212

Merged
merged 1 commit into from
Sep 29, 2021

Conversation

Sl1mb0
Copy link
Contributor

@Sl1mb0 Sl1mb0 commented Sep 24, 2021

Addresses #89206

r? @jyn514

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 24, 2021
src/bootstrap/setup.rs Outdated Show resolved Hide resolved
src/bootstrap/setup.rs Show resolved Hide resolved
let build = TargetSelection::from_user(&env!("BUILD_TRIPLE"));
let stage_path = ["build", build.rustc_target_arg(), "stage1"].join("/");
let toolchain_link =
Command::new("rustup").args(&["toolchain", "link", "stage1", &stage_path[..]]).output();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized this will overwrite any existing toolchain - that seems kind of not great, it means we'll delete any existing toolchain with this name. @kinnison @rbtcollins is that intentional that it will silently overwrite?

I guess to avoid that we could explicitly run rustup toolchain list and see if it already has stage1.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually this command is executed by the user, so like most Unix-y commands, dispatches as silently and swiftly as possible, and alerting someone is a sign of failure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm - but ln -s (which you said in the original issue is basically what toolchain link does) gives an error if the file already exists:

$ ln -s x config.toml
ln: failed to create symbolic link 'config.toml': File exists

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reasonable~
But the other reason I mentioned links is due to problems due to OS control over when files may be edited/clobbered/removed, which has less to do with the particular link.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we have to create the directory in the first place, couldn't we just not link if it already exists?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we have to create the directory in the first place, couldn't we just not link if it already exists?

That doesn't seem great - that won't warn or setup the toolchain if the user has already run x.py build but not created a rustup toolchain.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is why @Mark-Simulacrum suggested using a stamp file. We could also extend this by adding configuration for it in the config.toml; e.g. toolchain-overwrite or something where the user can specify if they want to overwrite the toolchain every time they run x.py setup or not, and by default ./x.py setup will only do it once.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't think this needs additional configuration. We can just check if rustup toolchain list contains stage1.

src/bootstrap/setup.rs Show resolved Hide resolved
@jyn514 jyn514 added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 24, 2021
@jyn514 jyn514 changed the title Xpy toolchain link x.py: run rustup toolchain link in setup Sep 24, 2021
Copy link
Member

@workingjubilee workingjubilee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we considered the possible failure mode of a distro-installed Rustup that is later overshadowed by Rustup's normal /home/user/.{cargo,rustup} install process? Possibly that doesn't matter: maybe such a case would make one expect possible overshadowing.

Comment on lines 116 to 120
println!("WARNING: `rustup`failed to link stage 1 build to `stage1` toolchain");
println!(
"To manually link stage 1 build to `stage1` tool chain, run:\n
`rustup toolchain link stage1 build/{}/stage1`",
build.rustc_target_arg()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So... I could have missed something about the previous code, but at this point in the program, what proof do we have that the user has Rustup on their system that justifies emitting this message all "WARNING!" style, telling them about a binary that possibly isn't installed on their system? It is entirely possible that the user is bootstrapping Rust on their system for the first time, Rustup is not available for this target, and we should be careful to neither alarm nor distract them. Something gentler and softer-voiced seems appropriate.

Copy link
Contributor Author

@Sl1mb0 Sl1mb0 Sep 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If rustup is installed:

Linking `stage1` toolchain

If not:

`rustup` is not installed
Unable to link `stage1` toolchain

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be fine to only warn if the profile isn't "user" (which is what distros should be using); for everyone else they would find rustup really useful for rust-lang/rust even if they wouldn't use it normally.

Copy link
Member

@workingjubilee workingjubilee Sep 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's mostly about framing: gently observing that they could do so is fine, as long as it doesn't appear on every invocation after, if they decide they don't want to use it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@workingjubilee yup - this is only running once when you first run setup, it's never repeated.

src/bootstrap/setup.rs Show resolved Hide resolved
let build = TargetSelection::from_user(&env!("BUILD_TRIPLE"));
let stage_path = ["build", build.rustc_target_arg(), "stage1"].join("/");
let toolchain_link =
Command::new("rustup").args(&["toolchain", "link", "stage1", &stage_path[..]]).output();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually this command is executed by the user, so like most Unix-y commands, dispatches as silently and swiftly as possible, and alerting someone is a sign of failure.

@jyn514
Copy link
Member

jyn514 commented Sep 24, 2021

Have we considered the possible failure mode of a distro-installed Rustup that is later overshadowed by Rustup's normal /home/user/.{cargo,rustup} install process?

I'm not sure how this could go wrong. If there's a distro-installed rustup, it could run toolchain link just like a binary-installed rustup. toolchain link doesn't change the default, just add additional toolchains.

@workingjubilee
Copy link
Member

I more meant user surprisal due to the overshadowing and the linked toolchain "disappearing", if that mattered.

@jyn514
Copy link
Member

jyn514 commented Sep 24, 2021

I clarified offline - the way this can go wrong is:

  1. Someone has a version of rustup installed from their distro (in /usr/bin/rustup or wherever).
  2. They build rust from source for reasons and x.py setup runs rustup toolchain link stage1.
  3. They install a new version of rustup so they can download new toolchains.
  4. The linked toolchain disappears.

Given that in this (somewhat convoluted) scenario they'll be able to easily rerun toolchain link or just run rustup install, I'm ok with the toolchain disappearing in that case; I don't see a way to avoid it and the benefit here massively outweighs the risk IMO.

src/bootstrap/setup.rs Outdated Show resolved Hide resolved
src/bootstrap/setup.rs Outdated Show resolved Hide resolved
src/bootstrap/setup.rs Outdated Show resolved Hide resolved
src/bootstrap/setup.rs Outdated Show resolved Hide resolved
src/bootstrap/setup.rs Outdated Show resolved Hide resolved
src/bootstrap/setup.rs Outdated Show resolved Hide resolved
@rust-log-analyzer

This comment has been minimized.

@jyn514 jyn514 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 24, 2021
Copy link
Member

@jyn514 jyn514 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good to me :) I just have a couple last nits.

@Mark-Simulacrum do you want to take a look before I approve? No problem if not, this seems pretty low risk.

Comment on lines 146 to 120
} else {
println!(
"Added `stage1` rustup toolchain; try `cargo +stage1 build` on a separate rust project to run a newly-built toolchain"
);
}
} else {
println!(
"`stage1` toolchain already linked; not attempting to link `stage1` toolchain",
);
}
} else {
println!(
"`rustup` failed to list current toolchains; not attempting to link `stage1` toolchain"
);
}
} else if profile != Profile::User {
println!("`rustup` is not installed; cannot link `stage1` toolchain");
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This long list of else blocks makes it hard to tell which error corresponds with which check IMO - could you instead use a separate functions and early returns so it's more clear? Something like this:

    maybe_create_rustup_toolchain();
// ...
}

fn maybe_create_rustup_toolchain() {
     if !rustup_installed || !stage_dir_exists {
        // ...
        return
     }
     // ...
}

src/bootstrap/setup.rs Show resolved Hide resolved
@jyn514 jyn514 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 25, 2021
@Mark-Simulacrum
Copy link
Member

No, happy to let you own the review here. Can always adjust later in future PRs if it becomes necessary :)

@jyn514
Copy link
Member

jyn514 commented Sep 26, 2021

@bors r+

This is awesome, thank you! ❤️

@bors
Copy link
Contributor

bors commented Sep 26, 2021

📌 Commit 91f77f7b8850d10b263fc6f35ed1781933c9eb37 has been approved by jyn514

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 26, 2021
@Mark-Simulacrum
Copy link
Member

@bors r-

Please squash the commits here to keep history a little cleaner.

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 26, 2021
Fixed types

Add checks for rustup and if toolchain is linked

Fortified rustup/directory checks; made other suggested changes

Added check for output status

Remove output of rustup from console

Made suggested change

Deleted confusing comment

Fixed compiler error; removed extra declaration

Refactored to smaller components; made suggested changes

Automate toolchain linking for stage 1 builds
@jyn514
Copy link
Member

jyn514 commented Sep 28, 2021

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Sep 28, 2021

📌 Commit adbb608 has been approved by jyn514

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 28, 2021
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this pull request Sep 28, 2021
x.py: run `rustup toolchain link` in setup

Addresses rust-lang#89206

r? `@jyn514`
This was referenced Sep 28, 2021
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 29, 2021
…laumeGomez

Rollup of 8 pull requests

Successful merges:

 - rust-lang#87260 (Libgccjit codegen)
 - rust-lang#89212 (x.py: run `rustup toolchain link` in setup)
 - rust-lang#89233 (Hide `<...> defined here` note if the source is not available)
 - rust-lang#89235 (make junit output more consistent with default format)
 - rust-lang#89255 (Fix incorrect disambiguation suggestion for associated items)
 - rust-lang#89276 (Fix the population of the `union.impls` field)
 - rust-lang#89283 (Add regression test for issue rust-lang#83564)
 - rust-lang#89318 (rustc_session: Remove lint store from `Session`)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 91da29f into rust-lang:master Sep 29, 2021
@rustbot rustbot added this to the 1.57.0 milestone Sep 29, 2021
@Sl1mb0 Sl1mb0 deleted the xpy-toolchain-link branch October 5, 2021 17:17
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 6, 2022
…etup, r=Mark-Simulacrum

Fix linking stage1 toolchain in `./x.py setup`

Closes [92319](rust-lang#92319)

Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by `@Sl1mb0.`

We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497).

cc `@jyn514` and `@Mark-Simulacrum` as they were active in rust-lang#89206
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 7, 2022
…etup, r=Mark-Simulacrum

Fix linking stage1 toolchain in `./x.py setup`

Closes [92319](rust-lang#92319)

Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by ``@Sl1mb0.``

We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497).

cc ``@jyn514`` and ``@Mark-Simulacrum`` as they were active in rust-lang#89206
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 7, 2022
…etup, r=Mark-Simulacrum

Fix linking stage1 toolchain in `./x.py setup`

Closes [92319](rust-lang#92319)

Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by ```@Sl1mb0.```

We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497).

cc ```@jyn514``` and ```@Mark-Simulacrum``` as they were active in rust-lang#89206
m-ou-se added a commit to m-ou-se/rust that referenced this pull request Feb 7, 2022
…etup, r=Mark-Simulacrum

Fix linking stage1 toolchain in `./x.py setup`

Closes [92319](rust-lang#92319)

Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by ````@Sl1mb0.````

We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497).

cc ````@jyn514```` and ````@Mark-Simulacrum```` as they were active in rust-lang#89206
m-ou-se added a commit to m-ou-se/rust that referenced this pull request Feb 7, 2022
…etup, r=Mark-Simulacrum

Fix linking stage1 toolchain in `./x.py setup`

Closes [92319](rust-lang#92319)

Fix linking stage1 toolchain in `./x.py setup`. I guess this can be considered a follow up to rust-lang#89212 by `````@Sl1mb0.`````

We create 2 directories and 1 file that are required by rustup to [link a custom toolchain from path](https://github.com/rust-lang/rustup/blob/5225e87a5d974ab5f1626bcb2a7b43f76ab883f0/src/toolchain.rs#L479-L497).

cc `````@jyn514````` and `````@Mark-Simulacrum````` as they were active in rust-lang#89206
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants