CI should exercise (subset of) tests under --stage 1

[pnkfelix]

Proposal

As discussed in rust-lang/rust#86366, x.py test by default runs the tests against a stage 1 build of the compiler.

However, the continuous integration (CI) used by bors only runs x.py test --stage 2. For good reasons; e.g., some of the tests are not guaranteed to work on a stage 1 build.

This leads to scenarios where the common case for a contributor can break, and there is no record of it in the CI or in the offending PR.

To combat this, we should identify the subset of the tests that we believe will work atop stage 1 builds, and add a CI process that tests them. This will help us catch cases like rust-lang/rust#86366 before they become problems for contributors.

• @jyn514 has suggested using the UI tests ({ui,rustdoc-ui}) as the initial subset that drives the stage1 tests.
• @wesleywiser noted that they rarely encounter issues on stage1 with the ui, debuginfo, codegen, codegen-units, incremental and mir-opt test suites.
• For an initial deployment, I suggest we start with ui and rustdoc-ui test suites. Representatives from T-infra said that the Linux builds should have enough slack to cover this for bors-only tests

Mentors or Reviewers

I'm willing to mentor this.

Process

The main points of the Major Change Process are as follows:

• File an issue describing the proposal.
• A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
  • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
  • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
• Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

Comments

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[rustbot]

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

cc @rust-lang/compiler @rust-lang/compiler-contributors

[jyn514]

@rustbot second

[apiraino]

@rustbot label -final-comment-period +major-change-accepted