document keep-stage1 and also try to clarify #191

nikomatsakis · 2018-08-31T15:36:40Z

This documents the --keep-stage 1 workflow (thanks @Mark-Simulacrum !) and tries to clarify a bit how staging works.

cc @sunjay -- the new language is intended to clear up some of the confusion you had around why building --stage 1 src/libstd does more than build the standard library. What do you think?

sunjay · 2018-08-31T15:47:53Z

LGTM. Thanks for adding all of this to the guide! 😁🎉

ehuss

A semi-related question: I've been confused for a long time why some tests have various stage options (like ignore-stage1). Shouldn't the stage1 compiler be functionally identical to the stage2 compiler?

src/how-to-build-and-run.md

nikomatsakis · 2018-08-31T16:49:42Z

@ehuss I added a paragraph about that

ehuss · 2018-08-31T17:01:35Z

Oh, that makes sense. TYVM!

mark-i-m

LGTM :)

Thanks! I was wondering how to use --keep-stage 1 too.

I few minor nits/typos, then r=me

mark-i-m · 2018-08-31T17:40:41Z

src/how-to-build-and-run.md

-  we can build it again
-  with stage2 compiler which must be identical to itself,
-  unless something has broken.
+internally. The result is the compiling `rustc` is done in stages:


s/is the/is that/

mark-i-m · 2018-08-31T17:43:34Z

src/how-to-build-and-run.md

@@ -80,6 +86,8 @@ It is, in particular, very useful when you're doing some kind of
 "type-based refactoring", like renaming a method, or changing the
 signature of some function.

+<a name=command>


Could you please close this tag? Sometimes leaving open tags leads to weird rendering in mdbook.

mark-i-m · 2018-08-31T17:46:27Z

src/how-to-build-and-run.md

+build. The **full** rustc build (what you get if you just say `./x.py
+build`) has quite a few more steps:
+
+- Build librustc rustc with the stage1 compiler.


s/librustc rustc/rustc/ ?

mark-i-m · 2018-08-31T17:58:28Z

src/how-to-build-and-run.md

@@ -140,6 +160,63 @@ release: 1.25.0-dev
 LLVM version: 4.0
 ```

+<a name=workflow>


src/how-to-build-and-run.md

Mark-Simulacrum · 2018-08-31T23:43:53Z

src/how-to-build-and-run.md

+
+- Initial test run: `./x.py test -i --stage 1 src/test/ui`
+- Subsequent test run: `./x.py test -i --stage 1 src/test/ui --keep-stage 1`
+


I think it's good to note that the --keep-stage flag can be passed multiple times.

Furthermore, if you want guaranteed to work and optimal build times when changing just the compiler, the workflow I would suggest is: ./x.py build -i --stage 2 --keep-stage 0; the --stage 2 is optional. The effect of this command is that when the compiler is changed, we just rebuild it and you have a compiler that is now both fully capable (i.e., proc macros, sysroot fully populated) and you did not need to rebuild std at all. If, however, you wanted to have your changes take effect on the std that compiler uses, then that won't happen with this. But that's relatively rare.

and you did not need to rebuild std at all

I don't think such a compiler can possibly be guaranteed to work. I guess what you mean is "if you want a compiler that avoids the stage1 limitations"?

I decided not to add this text because I still don't really understand what --keep-stage 0 even means. :) Does it mean that we copy the stdlib from the beta?

I guess in that case maybe we always want to suggest it, unless you are editing the stdlib?

cc @Mark-Simulacrum

src/how-to-build-and-run.md

nikomatsakis · 2018-09-07T15:18:28Z

I'm pulling out this conversation about --keep-stage 0 etc since GitHub wants to hide it. @Mark-Simulacrum can you clarify?

I decided not to add this text because I still don't really understand what --keep-stage 0 even means. :) Does it mean that we copy the stdlib from the beta? I guess in that case maybe we always want to suggest it, unless you are editing the stdlib?

Mark-Simulacrum · 2018-09-08T01:04:23Z

I think there's a few workflows to consider:

Modifying std, with the intent of trying out some other user-space (i.e., not compiler-space) code. I believe the fastest way to do this today, when iterating, is: x.py build --stage 1 --keep-stage 0 src/libstd.

This will build the compiler once and only once
That compiler will then build a standard library and be mostly usable -- can build most code, but no proc macros currently
The compiler produced is build/<build triple>/stage1/bin/rustc

Modifying std, with intent to see results in the compiler. If you don't care about userspace -- which includes tests -- then x.py build --stage 1 src/libstd --keep-stage 1 will be the fastest way to start and iterate. This will rebuild std, test, and the compiler, but it won't rebuild std more than once. If you do care about userspace seeing the std changes, you'll need to either remove the --keep-stage 1 argument (and pay for two std rebuilds), or, if you're willing to invest more time upfront, then you can run x.py build --stage 2 --keep-stage 0 instead. This will build a fully working compiler, building std, test, and rustc once each time you change std, and produces a compiler capable of supporting proc macros.

Modifying the compiler:

If you need proc macros, you'll need to invest more time up-front. Use x.py build --keep-stage 0 src/librustc_codegen_llvm. This will, on each run, attempt to build std/test/rustc with the stage1/bin/rustc compiler. Since we're only changing rustc, this is quite fast.

If you don't need proc macros, and aren't changing metadata or other ABI details, you can invest less time up-front. Use x.py build --keep-stage 1 src/libstd. This will build everything as per normal but skips the std build in stage 1, just reusing "old" artifacts.

In general, any command run with --keep-stage N should have a preceding --stage N run with other arguments being the same, otherwise you will likely fail to produce a working compiler due to the artifacts being stale or nonexistent.

I'm happy to clarify further, not sure if I've properly explained or even explained in the wrong direction -- let me know what would be best.

nikomatsakis · 2018-09-11T16:03:36Z

My inclination is to merge this as is and file a follow-up issue with a link to @Mark-Simulacrum's comment. I don't think I have the time to try and write up more detailed notes here (and I also still feel sort of unclear about what anything other than --keep-stage 1 actually does)

nikomatsakis · 2018-09-11T20:25:15Z

Filed #201

nikomatsakis requested a review from mark-i-m August 31, 2018 15:36

sunjay approved these changes Aug 31, 2018

View reviewed changes

ehuss reviewed Aug 31, 2018

View reviewed changes

src/how-to-build-and-run.md Outdated Show resolved Hide resolved

src/how-to-build-and-run.md Outdated Show resolved Hide resolved

mark-i-m approved these changes Aug 31, 2018

View reviewed changes

Mark-Simulacrum reviewed Aug 31, 2018

View reviewed changes

dwijnand reviewed Sep 7, 2018

View reviewed changes

src/how-to-build-and-run.md Outdated Show resolved Hide resolved

dwijnand reviewed Sep 7, 2018

View reviewed changes

src/how-to-build-and-run.md Outdated Show resolved Hide resolved

mark-i-m mentioned this pull request Sep 11, 2018

issue-130 copy contents related x.py from rust-forge to rustc-guide #195

Merged

nikomatsakis added 8 commits September 11, 2018 12:02

document keep-stage1

a2ad0a8

wrap long lines

7d87e3c

note about why stage1/stage2 are different

4ab5fec

fix typos

4a6c34a

fix typo, add link

8cbf9d0

fix some more typos

576581d

remove silly note:

4490acd

wrap long lines

69cdc0a

nikomatsakis force-pushed the keep-stage branch from 80f26c7 to 69cdc0a Compare September 11, 2018 16:03

wrap lines in running.md

b655878

nikomatsakis mentioned this pull request Sep 11, 2018

explain in more depth about keep-stage #201

Closed

nikomatsakis merged commit 1fa5685 into rust-lang:master Sep 11, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

document keep-stage1 and also try to clarify #191

document keep-stage1 and also try to clarify #191

nikomatsakis commented Aug 31, 2018

sunjay commented Aug 31, 2018

ehuss left a comment

nikomatsakis commented Aug 31, 2018

ehuss commented Aug 31, 2018

mark-i-m left a comment

mark-i-m Aug 31, 2018

mark-i-m Aug 31, 2018

mark-i-m Aug 31, 2018

mark-i-m Aug 31, 2018

Mark-Simulacrum Aug 31, 2018

nikomatsakis Sep 7, 2018

nikomatsakis Sep 7, 2018

nikomatsakis Sep 7, 2018

nikomatsakis commented Sep 7, 2018

Mark-Simulacrum commented Sep 8, 2018

nikomatsakis commented Sep 11, 2018 •

edited

Loading

nikomatsakis commented Sep 11, 2018


		- Initial test run: `./x.py test -i --stage 1 src/test/ui`
		- Subsequent test run: `./x.py test -i --stage 1 src/test/ui --keep-stage 1`

document keep-stage1 and also try to clarify #191

document keep-stage1 and also try to clarify #191

Conversation

nikomatsakis commented Aug 31, 2018

sunjay commented Aug 31, 2018

ehuss left a comment

Choose a reason for hiding this comment

nikomatsakis commented Aug 31, 2018

ehuss commented Aug 31, 2018

mark-i-m left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nikomatsakis commented Sep 7, 2018

Mark-Simulacrum commented Sep 8, 2018

nikomatsakis commented Sep 11, 2018 • edited Loading

nikomatsakis commented Sep 11, 2018

nikomatsakis commented Sep 11, 2018 •

edited

Loading