Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download #93885

Merged
merged 1 commit into from
Feb 13, 2022

Conversation

Badel2
Copy link
Contributor

@Badel2 Badel2 commented Feb 11, 2022

I got an error when trying to build the compiler using an old commit, and it turns out it was because the option download-ci-llvm was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem.

To reproduce my error:

git checkout 8d7707f3c4f72e6eb334d897354beca692b265d1
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404 

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
Build completed unsuccessfully in 0:00:46

This is my config.toml:

# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[rust]
debug = true

To reproduce an error with this branch:

Change line 618 of bootstrap.py to

        url = "rustc-builds-error404/{}".format(llvm_sha)

Delete llvm and cached tarball, and set llvm.download-ci-llvm=true in config.toml.

./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404 

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
error: failed to download llvm from ci
help: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml:

[llvm]
download-ci-llvm = false

Build completed unsuccessfully in 0:00:01

Regarding the implementation, I expected to be able to use a try/catch block in _download_ci_llvm, but the run function calls sys.exit instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.

@rust-highfive
Copy link
Collaborator

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 11, 2022
@Mark-Simulacrum
Copy link
Member

@bors r+ rollup

This seems pretty positive to me.

@bors
Copy link
Contributor

bors commented Feb 11, 2022

📌 Commit dfa9d64 has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 11, 2022
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 12, 2022
…rk-Simulacrum

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download

I got an error when trying to build the compiler using an old commit, and it turns out it was because the option `download-ci-llvm` was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem.

To reproduce my error:

```
git checkout 8d7707f
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
Build completed unsuccessfully in 0:00:46
```

This is my `config.toml`:

```
# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[rust]
debug = true
```

To reproduce an error with this branch:

Change line 618 of bootstrap.py to
```
        url = "rustc-builds-error404/{}".format(llvm_sha)
```

Delete llvm and cached tarball, and set `llvm.download-ci-llvm=true` in config.toml.

```
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
error: failed to download llvm from ci
help: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml:

[llvm]
download-ci-llvm = false

Build completed unsuccessfully in 0:00:01
```

Regarding the implementation, I expected to be able to use a try/catch block in `_download_ci_llvm`, but the `run` function calls `sys.exit` instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.
@@ -178,6 +179,8 @@ def run(args, verbose=False, exception=False, is_bootstrap=False, **kwargs):
code = ret.wait()
if code != 0:
err = "failed to run: " + ' '.join(args)
if help_on_error is not None:
err = f"{err}\n{help_on_error}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately we cannot use f-strings, which are Python 3.6+. x.py is currently required to be compatible with Python 2.x though there are discussions about bumping it to Python 3 (Zulip thread).

This caused the rollup failure in #93929.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thank you.

@matthiaskrgr
Copy link
Member

@bors r-

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 12, 2022
@Badel2 Badel2 force-pushed the error-download-ci-llvm branch from dfa9d64 to 410145e Compare February 12, 2022 13:23
@Mark-Simulacrum
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Feb 12, 2022

📌 Commit 410145e has been approved by Mark-Simulacrum

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 12, 2022
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 12, 2022
…rk-Simulacrum

bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download

I got an error when trying to build the compiler using an old commit, and it turns out it was because the option `download-ci-llvm` was implicitly set to true. So this pull request tries to add a help message for other people that may run into the same problem.

To reproduce my error:

```
git checkout 8d7707f
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmp8g13rb4n https://ci-artifacts.rust-lang.org/rustc-builds/db002a06ae9154a35d410550bc5132df883d7baa/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
Build completed unsuccessfully in 0:00:46
```

This is my `config.toml`:

```
# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[rust]
debug = true
```

To reproduce an error with this branch:

Change line 618 of bootstrap.py to
```
        url = "rustc-builds-error404/{}".format(llvm_sha)
```

Delete llvm and cached tarball, and set `llvm.download-ci-llvm=true` in config.toml.

```
./x.py test
[...]
spurious failure, trying again
downloading https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

failed to run: curl -# -y 30 -Y 10 --connect-timeout 30 --retry 3 -Sf -o /tmp/tmpesl1ydvo https://ci-artifacts.rust-lang.org/rustc-builds-error404/719b04ca99be0c78e09a8ec5e2eda082a5d8ccae/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
error: failed to download llvm from ci
help: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-ci-llvm` in config.toml:

[llvm]
download-ci-llvm = false

Build completed unsuccessfully in 0:00:01
```

Regarding the implementation, I expected to be able to use a try/catch block in `_download_ci_llvm`, but the `run` function calls `sys.exit` instead of raising an exception so that's not possible. Also, suggestions for better wording of the help message are welcome.
bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 13, 2022
…askrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#89926 (make `Instant::{duration_since, elapsed, sub}` saturating and remove workarounds)
 - rust-lang#90532 (More informative error message for E0015)
 - rust-lang#93810 (Improve chalk integration)
 - rust-lang#93851 (More practical examples for `Option::and_then` & `Result::and_then`)
 - rust-lang#93885 (bootstrap.py: Suggest disabling download-ci-llvm option if url fails to download)
 - rust-lang#93886 (Stabilise inherent_ascii_escape (FCP in rust-lang#77174))
 - rust-lang#93930 (add link to format_args! when mention it in docs)
 - rust-lang#93936 (Couple of driver cleanups)
 - rust-lang#93944 (Don't relabel to a team if there is already a team label)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit b90a369 into rust-lang:master Feb 13, 2022
@rustbot rustbot added this to the 1.60.0 milestone Feb 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants