-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rustbuild: Compile rustc twice, not thrice #38631
Conversation
(rust_highfive has picked a reviewer for you, use r? to override) |
--enable-full-bootstrap | ||
ENV RUST_CHECK_TARGET "" | ||
RUN mkdir /tmp/obj | ||
RUN chmod 777 /tmp/obj |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: ccache
and /tmp/obj
are not removed (cc #38632)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops, indeed!
74d6815
to
2f5b4ed
Compare
2f5b4ed
to
d8fe45f
Compare
This commit switches the rustbuild build system to compiling the compiler twice for a normal bootstrap rather than the historical three times. Rust is a bootstrapped language which means that a previous version of the compiler is used to build the next version of the compiler. Over time, however, we change many parts of compiler artifacts such as the metadata format, symbol names, etc. These changes make artifacts from one compiler incompatible from another compiler. Consequently if a compiler wants to be able to use some artifacts then it itself must have compiled the artifacts. Historically the rustc build system has achieved this by compiling the compiler three times: * An older compiler (stage0) is downloaded to kick off the chain. * This compiler now compiles a new compiler (stage1) * The stage1 compiler then compiles another compiler (stage2) * Finally, the stage2 compiler needs libraries to link against, so it compiles all the libraries again. This entire process amounts in compiling the compiler three times. Additionally, this process always guarantees that the Rust source tree can compile itself because the stage2 compiler (created by a freshly created compiler) would successfully compile itself again. This property, ensuring Rust can compile itself, is quite important! In general, though, this third compilation is not required for general purpose development on the compiler. The third compiler (stage2) can reuse the libraries that were created during the second compile. In other words, the second compilation can produce both a compiler and the libraries that compiler will use. These artifacts *must* be compatible due to the way plugins work today anyway, and they were created by the same source code so they *should* be compatible as well. So given all that, this commit switches the default build process to only compile the compiler three times, avoiding this third compilation by copying artifacts from the previous one. Along the way a new entry in the Travis matrix was also added to ensure that our full bootstrap can succeed. This entry does not run tests, though, as it should not be necessary. To restore the old behavior of a full bootstrap (three compiles) you can either pass: ./configure --enable-full-bootstrap or if you're using config.toml: [build] full-bootstrap = true Overall this will hopefully be an easy 33% win in build times of the compiler. If we do 33% less work we should be 33% faster! This in turn should affect cycle times and such on Travis and AppVeyor positively as well as making it easier to work on the compiler itself.
d8fe45f
to
7046fea
Compare
@bors r+ |
📌 Commit 7046fea has been approved by |
A small PR message nit:
three -> two |
Oops, indeed! |
rustbuild: Compile rustc twice, not thrice This commit switches the rustbuild build system to compiling the compiler twice for a normal bootstrap rather than the historical three times. Rust is a bootstrapped language which means that a previous version of the compiler is used to build the next version of the compiler. Over time, however, we change many parts of compiler artifacts such as the metadata format, symbol names, etc. These changes make artifacts from one compiler incompatible from another compiler. Consequently if a compiler wants to be able to use some artifacts then it itself must have compiled the artifacts. Historically the rustc build system has achieved this by compiling the compiler three times: * An older compiler (stage0) is downloaded to kick off the chain. * This compiler now compiles a new compiler (stage1) * The stage1 compiler then compiles another compiler (stage2) * Finally, the stage2 compiler needs libraries to link against, so it compiles all the libraries again. This entire process amounts in compiling the compiler three times. Additionally, this process always guarantees that the Rust source tree can compile itself because the stage2 compiler (created by a freshly created compiler) would successfully compile itself again. This property, ensuring Rust can compile itself, is quite important! In general, though, this third compilation is not required for general purpose development on the compiler. The third compiler (stage2) can reuse the libraries that were created during the second compile. In other words, the second compilation can produce both a compiler and the libraries that compiler will use. These artifacts *must* be compatible due to the way plugins work today anyway, and they were created by the same source code so they *should* be compatible as well. So given all that, this commit switches the default build process to only compile the compiler two times, avoiding this third compilation by copying artifacts from the previous one. Along the way a new entry in the Travis matrix was also added to ensure that our full bootstrap can succeed. This entry does not run tests, though, as it should not be necessary. To restore the old behavior of a full bootstrap (three compiles) you can either pass: ./configure --enable-full-bootstrap or if you're using config.toml: [build] full-bootstrap = true Overall this will hopefully be an easy 33% win in build times of the compiler. If we do 33% less work we should be 33% faster! This in turn should affect cycle times and such on Travis and AppVeyor positively as well as making it easier to work on the compiler itself.
This commit switches the rustbuild build system to compiling the
compiler twice for a normal bootstrap rather than the historical three
times.
Rust is a bootstrapped language which means that a previous version of
the compiler is used to build the next version of the compiler. Over
time, however, we change many parts of compiler artifacts such as the
metadata format, symbol names, etc. These changes make artifacts from
one compiler incompatible from another compiler. Consequently if a
compiler wants to be able to use some artifacts then it itself must have
compiled the artifacts.
Historically the rustc build system has achieved this by compiling the
compiler three times:
compiles all the libraries again.
This entire process amounts in compiling the compiler three times.
Additionally, this process always guarantees that the Rust source tree
can compile itself because the stage2 compiler (created by a freshly
created compiler) would successfully compile itself again. This
property, ensuring Rust can compile itself, is quite important!
In general, though, this third compilation is not required for general
purpose development on the compiler. The third compiler (stage2) can
reuse the libraries that were created during the second compile. In
other words, the second compilation can produce both a compiler and the
libraries that compiler will use. These artifacts must be compatible
due to the way plugins work today anyway, and they were created by the
same source code so they should be compatible as well.
So given all that, this commit switches the default build process to
only compile the compiler two times, avoiding this third compilation
by copying artifacts from the previous one. Along the way a new entry in
the Travis matrix was also added to ensure that our full bootstrap can
succeed. This entry does not run tests, though, as it should not be
necessary.
To restore the old behavior of a full bootstrap (three compiles) you can
either pass:
or if you're using config.toml:
Overall this will hopefully be an easy 33% win in build times of the
compiler. If we do 33% less work we should be 33% faster! This in turn
should affect cycle times and such on Travis and AppVeyor positively as
well as making it easier to work on the compiler itself.