ci: Attempt to skip a full rustc compile on dist* #61212
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently when we're preparing cross-compiled compilers it can take quite some time because we have to build the compiler itself three different times. The first is the normal bootstrap, the second is a second build for the build platform, and the third is the actual target architecture compiler. The second compiler was historically built exclusively for procedural macros, and long ago we didn't actually need it. This commit tries out avoiding that second compiled compiler, meaning we only compile rustc for the build platform only once. Some local testing shows that this is promising, but bors is of course the ultimate test!
|
(rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
|
@bors r+ p=1 This should also give us a big performance boost on the current Travis and AppVeyor setup. |
|
📌 Commit ad52c77 has been approved by |
bors
added
S-waiting-on-bors
|
@bors rollup=never |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
bors
added
S-waiting-on-review
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
bors
added
S-waiting-on-bors
|
@bors: r- Hm looking at the logs there's something else to remove, need to keep digging. |
bors
added
S-waiting-on-author
This comment has been minimized.
This comment has been minimized.
This commit furthers the previous one to ensure that we don't build an extra stage of the compiler in CI. A test has been added to rustbuild to ensure that this doesn't regress, and then in debugging this test it was hunted down that the `dist::Std` target was the one erroneously pulling in the wrong compiler. The `dist::Std` step was updated to instead account for the "full bootstrap" or not flag, ensuring that the correct compiler for compiling the final standard library was used. This was another use of the `force_use_stage1` function which was in theory supposed to be pretty central, so existing users were all evaluated and a new function, `Builder::compiler_for`, was introduced. All existing users of `force_use_stage1` have been updated to use `compiler_for`, where the semantics of `compiler_for` are similar to that of `compiler` except that it doesn't guarantee the presence of a sysroot for the arguments passed (as they may be modified). Perhaps one day we can unify `compiler` and `compiler_for`, but the usage of `Builder::compiler` is so ubiquitous it would take quite some time to evaluate whether each one needs the sysroot or not, so it's hoped that can be done in parallel.
|
Ok the last two commits I believe actually fix the issue, re-r? @pietroalbini |
pietroalbini
added
S-waiting-on-review
|
This looks good! Let's try again. @bors r+ p=1 rollup=never |
|
📌 Commit 7b362bb has been approved by |
bors
added
S-waiting-on-bors
|
@bors p=19 |
|
⌛ Testing commit 7b362bb with merge 85774b4b8d5b7928b40e338268ba7caa9a1d46ba... |
|
💔 Test failed - checks-travis |
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
pietroalbini
added
S-waiting-on-author
|
Timeout:
|
|
Ugh, it almost finished :/ @bors retry |
bors
added
S-waiting-on-bors
bors
added a commit
that referenced
this pull request
May 30, 2019
ci: Attempt to skip a full rustc compile on dist* Currently when we're preparing cross-compiled compilers it can take quite some time because we have to build the compiler itself three different times. The first is the normal bootstrap, the second is a second build for the build platform, and the third is the actual target architecture compiler. The second compiler was historically built exclusively for procedural macros, and long ago we didn't actually need it. This commit tries out avoiding that second compiled compiler, meaning we only compile rustc for the build platform only once. Some local testing shows that this is promising, but bors is of course the ultimate test!
|
☀️ Test successful - checks-travis, status-appveyor |
alexcrichton
added a commit
to alexcrichton/rust
that referenced
this pull request
May 30, 2019
Since rust-lang#61212 we've been timing out on OSX, and this looks to be because we're building tools like Cargo and the RLS twice instead of once. This turns out to be a slight bug in our configuration. CI builders using the `RUST_CHECK_TARGET` directive actually execute `make all` just before their acual target. In `make all` we're building a stage2 cargo, and then in `make dist` we're building a stage1 cargo. Other builders use `SCRIPT` which provides explicit control over what `x.py` script, for example, is used to execute the build. This moves almost all targets to using `SCRIPT` to ensure that we're explicitly specifying what's being built where. Additionally this updates the logic of `RUST_CHECK_TARGET` to remove the pre-flight tidy as well as the pre-flight `make all`. The system LLVM builder (run on PRs) now explicitly runs tidy first and then runs the rest of the test suite.
bors
added a commit
that referenced
this pull request
May 30, 2019
ci: Favor SCRIPT instead of RUST_CHECK_TARGET Since #61212 we've been timing out on OSX, and this looks to be because we're building tools like Cargo and the RLS twice instead of once. This turns out to be a slight bug in our configuration. CI builders using the `RUST_CHECK_TARGET` directive actually execute `make all` just before their acual target. In `make all` we're building a stage2 cargo, and then in `make dist` we're building a stage1 cargo. Other builders use `SCRIPT` which provides explicit control over what `x.py` script, for example, is used to execute the build. This moves almost all targets to using `SCRIPT` to ensure that we're explicitly specifying what's being built where. Additionally this updates the logic of `RUST_CHECK_TARGET` to remove the pre-flight tidy as well as the pre-flight `make all`. The system LLVM builder (run on PRs) now explicitly runs tidy first and then runs the rest of the test suite.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!