Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Kobzol · 2025-08-20T15:15:22Z

This PR cleans up a bunch of test steps and adds metadata to them. I didn't yet touch the most complicated step (CompileTest), I'm leaving that for another PR.

Testing anything on stage 0 is only possible for compiletest and with build.allow-compiletest-stage0. Testing anything else on stage 0 will either produce a nice error or crash with a stage being subtracted below zero.

r? @jieyouxu

try-job: dist-x86_64-linux
try-job: aarch64-gnu
try-job: arm-android
try-job: x86_64-gnu-llvm-20*
try-job: x86_64-msvc-*
try-job: aarch64-apple
try-job: test-various

rustbot · 2025-08-20T15:16:02Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Kobzol · 2025-08-20T15:19:54Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

Kobzol · 2025-08-20T17:37:55Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-bors · 2025-08-20T17:51:18Z

💔 Test for 06dca67 failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

Kobzol · 2025-08-20T17:53:44Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple

rust-bors · 2025-08-20T18:01:48Z

💔 Test for 71ba459 failed: CI. Failed jobs:

try - aarch64-apple (web logs, extended logs)
try - aarch64-gnu (web logs, extended logs)
try - arm-android (web logs, extended logs)
try - dist-x86_64-linux (web logs, extended logs)
(and 8 other jobs)

rustbot · 2025-08-30T13:42:28Z

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Kobzol · 2025-08-30T14:39:41Z

@bors try

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) try-job: dist-x86_64-linux try-job: aarch64-gnu try-job: arm-android try-job: `x86_64-gnu-llvm-20*` try-job: `x86_64-msvc-*` try-job: aarch64-apple try-job: test-various

rust-bors · 2025-08-30T17:47:15Z

☀️ Try build successful (CI)
Build commit: 3d5b72c (3d5b72c31733c321331b65ec5377bf47f86e4b45, parent: e95db591a4550e28ad92660b753ad85b89271882)

Kobzol · 2025-08-30T19:24:55Z

compiletest stage 1 => build stage 1, test it
x test compiler stage 1 => build stage 1 in test mode, run its unit + doc tests

@rustbot ready

jieyouxu

Thanks

View changes since this review

jieyouxu · 2025-09-01T01:00:44Z

@bors r+ rollup=never

bors · 2025-09-01T01:00:46Z

📌 Commit 5bc9308 has been approved by jieyouxu

It is now in the queue for this repository.

bors · 2025-09-01T01:30:31Z

⌛ Testing commit 5bc9308 with merge 828e45a...

bors · 2025-09-01T04:36:32Z

☀️ Test successful - checks-actions
Approved by: jieyouxu
Pushing 828e45a to master...

github-actions · 2025-09-01T04:39:36Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 07d246f (parent) -> 828e45a (this PR)

Test differences

Show 34592 test diffs

Stage 0

base_n::tests::test_to_base: [missing] -> pass (J4)
errors::verify_builtin_macros_alloc_must_statics_24: [missing] -> pass (J4)
errors::verify_codegen_ssa_invalid_monomorphization_simd_input_86: [missing] -> pass (J4)
errors::verify_expand_proc_macro_back_compat_34: [missing] -> pass (J4)
errors::verify_metadata_no_multiple_alloc_error_handler_21: [missing] -> pass (J4)
errors::verify_mir_build_const_param_in_pattern_1: [missing] -> pass (J4)
errors::verify_parse_dyn_after_mut_138: [missing] -> pass (J4)
errors::verify_passes_both_ffi_const_and_pure_45: [missing] -> pass (J4)
parser::tests::triple_exact_overlap: [missing] -> pass (J4)
parser::tokenstream::tests::test_is_empty: [missing] -> pass (J4)
spec::base::apple::tests::simulator_targets_set_env: [missing] -> pass (J4)
spec::tests::i686_unknown_uefi: [missing] -> pass (J4)
spec::tests::powerpc_unknown_linux_gnuspe: [missing] -> pass (J4)
spec::tests::powerpc_unknown_linux_muslspe: [missing] -> pass (J4)

Stage 1

condvar::nonpoison_smoke: pass -> [missing] (J0)
io::buffered::tests::partial_multiline_buffering: pass -> [missing] (J0)
io::cursor::tests::test_mem_mut_writer: pass -> [missing] (J0)
io::tests::read_buf_exact: pass -> [missing] (J0)
mpmc::receiver_cloning: pass -> [missing] (J0)
mpsc_sync::smoke_shared_port_gone2: pass -> [missing] (J0)
thread::tests::test_named_thread: pass -> [missing] (J0)
iter::bench_skip_ref_sum: pass -> [missing] (J1)
net::addr_parser::bench_parse_socketaddr_v6: pass -> [missing] (J1)
slice::starts_with_single_element: pass -> [missing] (J1)
sort::tests::stable::correct_i32_random_z2: pass -> [missing] (J1)
array::iterator_drops: pass -> [missing] (J2)
boxed::panic_no_leak: pass -> [missing] (J2)
cell::ref_and_refmut_have_sensible_show: pass -> [missing] (J2)
collections::btree::map::tests::test_check_ord_chaos: pass -> [missing] (J2)
const_ptr::write: pass -> [missing] (J2)
floats::is_normal::test_f64: pass -> [missing] (J2)
hash::sip::test_hash_usize: pass -> [missing] (J2)
iter::adapters::array_chunks::test_iterator_array_chunks_size_hint: pass -> [missing] (J2)
iter::adapters::chain::test_chain_fold: pass -> [missing] (J2)
rc::test_rc_cyclic_with_two_ref: pass -> [missing] (J2)
rc::test_unique_rc_with_alloc_drops_contents: pass -> [missing] (J2)
slice::test_pop: pass -> [missing] (J2)
slice::test_shrink_to_fit: pass -> [missing] (J2)
sort::tests::unstable::deterministic_string_random_z1: pass -> [missing] (J2)
str::test_repeat: pass -> [missing] (J2)
string::test_str_concat: pass -> [missing] (J2)
unicode::version: pass -> [missing] (J2)
edit_distance::tests::test_method_name_similarity_score: pass -> [missing] (J4)
errors::verify_codegen_ssa_multiple_instruction_set_22: pass -> [missing] (J4)
errors::verify_parse_macro_invocation_visibility_160: pass -> [missing] (J4)
errors::verify_passes_rustc_legacy_const_generics_index_50: pass -> [missing] (J4)
errors::verify_resolve_cannot_glob_import_possible_crates_34: pass -> [missing] (J4)
lints::verify_lint_identifier_uncommon_codepoints_76: pass -> [missing] (J4)
maybe_transmutable::tests::bool::should_permit_identity_transmutation_tree: pass -> [missing] (J4)
opaque::tests::test_usize: pass -> [missing] (J4)
scope::tests::panic_propagate_spawn: pass -> [missing] (J4)
session_diagnostics::verify_attr_parsing_empty_confusables_34: pass -> [missing] (J4)
session_diagnostics::verify_attr_parsing_link_ordinal_out_of_range_43: pass -> [missing] (J4)
sort::tests::stable::panic_retain_orig_set_cell_i32_descending: ignore -> [missing] (J5)
sort::tests::unstable::stability_legacy: ignore -> [missing] (J5)

Stage 2

ascii::is_ascii::short::case02_align_to: [missing] -> pass (J1)
ascii::long::case13_subtract_shifted_bool_match_range: [missing] -> pass (J1)
fmt::write_str_macro2: [missing] -> pass (J1)
fs::tests::dir_entry_methods: [missing] -> pass (J1)
net::tcp::tests::multiple_connect_serial: [missing] -> pass (J1)
slice::rotate_tiny_half_plus_one: [missing] -> pass (J1)
sort::tests::stable::correct_dyn_val_descending: [missing] -> pass (J1)
sort::tests::stable::correct_f128_descending: [missing] -> pass (J1)
sort::tests::stable::correct_u128_ascending: [missing] -> pass (J1)
sort::tests::stable::panic_observable_is_less_saw_mixed: [missing] -> pass (J1)
sort::tests::stable::violate_ord_retain_orig_set_cell_i32_random_d20: [missing] -> pass (J1)
str::bench_contains_bad_naive: [missing] -> pass (J1)
str::char_count::emoji_huge::case03_manual_char_len: [missing] -> pass (J1)
str::char_count::en_small::case03_manual_char_len: [missing] -> pass (J1)
array::array_into_iter_fold: [missing] -> pass (J2)
array::array_rsplit_array_mut_out_of_bounds: [missing] -> pass (J2)
cell::ref_map_does_not_update_flag: [missing] -> pass (J2)
collections::linked_list::tests::test_append: [missing] -> pass (J2)
fmt::builders::debug_map::test_multiple_and_non_exhaustive: [missing] -> pass (J2)
io::borrowed_buf::advance_filled: [missing] -> pass (J2)
io::buffered::tests::bench_buffered_reader: [missing] -> pass (J2)
io::buffered::tests::line_write_all_error: [missing] -> pass (J2)
io::cursor::tests::test_buf_writer_vectored: [missing] -> pass (J2)
iter::adapters::step_by::test_iterator_step_by_nth_try_rfold: [missing] -> pass (J2)
iter::sources::test_successors: [missing] -> pass (J2)
lazy::once_cell_drop: [missing] -> pass (J2)
macros::cfg_select_options: [missing] -> pass (J2)
nonzero::nonzero_trailing_zeros: [missing] -> pass (J2)
num::test_boolu16: [missing] -> pass (J2)
ptr::test_ptr_from_raw_parts_in_const: [missing] -> pass (J2)
slice::slice_index::range_neg_width::index_fail: [missing] -> pass (J2)
slice::test_box_slice_clone: [missing] -> pass (J2)
slice::test_chunks_mut_nth: [missing] -> pass (J2)
sort::tests::unstable::violate_ord_retain_orig_set_cell_i32_random_z1: [missing] -> pass (J2)
str::not_double_ended_rsplitn: [missing] -> pass (J2)
sync::test_from_str: [missing] -> pass (J2)
vec_deque::test_range_end_overflow: [missing] -> pass (J2)
floats::minimum::test_f16: [missing] -> pass (J3)
floats::nan::test_f128: [missing] -> pass (J3)
floats::next_up::test_f16: [missing] -> pass (J3)
floats::rem_euclid::test_f16: [missing] -> pass (J3)
sort::tests::stable::panic_retain_orig_set_i32_descending: [missing] -> ignore (J5)
sort::tests::stable::self_cmp_string_random_d2: [missing] -> ignore (J5)
sys::args::windows::tests::empty: [missing] -> pass (J6)
sys::path::windows::tests::verbatim: [missing] -> pass (J6)

(and 17251 additional test diffs)

Additionally, 17241 doctest diffs were found. These are ignored, as they are noisy.

Job group index

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 828e45ad11ce4ab56dd64e93f1fb5dd8f0c0ae93 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

dist-aarch64-linux: 6204.9s -> 8446.9s (36.1%)
dist-aarch64-msvc: 5220.4s -> 6554.7s (25.6%)
x86_64-gnu-llvm-19: 2416.0s -> 2936.3s (21.5%)
aarch64-gnu-llvm-19-2: 2194.8s -> 2611.4s (19.0%)
x86_64-rust-for-linux: 2627.1s -> 3011.9s (14.6%)
x86_64-gnu-tools: 3350.1s -> 3814.8s (13.9%)
x86_64-gnu-distcheck: 5068.9s -> 5765.9s (13.8%)
x86_64-gnu-llvm-20-2: 5603.5s -> 6320.2s (12.8%)
i686-gnu-2: 5473.3s -> 6157.3s (12.5%)
pr-check-2: 2098.5s -> 2355.3s (12.2%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-09-01T06:05:35Z

Finished benchmarking commit (828e45a): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results (primary -1.2%, secondary 1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.3%	[1.3%, 1.3%]	1
Improvements ✅ (primary)	-1.2%	[-1.2%, -1.2%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-1.2%	[-1.2%, -1.2%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 468.707s -> 466.581s (-0.45%)
Artifact size: 388.59 MiB -> 388.59 MiB (-0.00%)

rustbot assigned jieyouxu Aug 20, 2025

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 20, 2025

This comment has been minimized.

Sign in to view

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

rustbot added A-CI Area: Our Github Actions CI T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Aug 20, 2025

This comment has been minimized.

Sign in to view

Kobzol force-pushed the bootstrap-test branch from 00e0ffb to c501499 Compare August 20, 2025 15:34

This comment has been minimized.

Sign in to view

Kobzol force-pushed the bootstrap-test branch from 8e5d2e3 to 9887c77 Compare August 20, 2025 16:52

This comment has been minimized.

Sign in to view

Kobzol added 6 commits August 30, 2025 15:28

Fix doclink

1591901

Fake nodejs in snapshot test

1c1d630

Fix tests and doclink

daaf0c9

Add test stage 2 snapshot tests

d2195c4

Fix stage 1 compiler tests

9de09ac

Do not run lint-docs tests in stage 1 x test by default

5bc9308

Kobzol force-pushed the bootstrap-test branch from dc19242 to 5bc9308 Compare August 30, 2025 13:42

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 30, 2025

jieyouxu approved these changes Sep 1, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 1, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Sep 1, 2025

bors merged commit 828e45a into rust-lang:master Sep 1, 2025
12 checks passed

rustbot added this to the 1.91.0 milestone Sep 1, 2025

This was referenced Sep 1, 2025

Add compiler-builtins to bootstrap #142106

Open

Add bootstrap.toml option to control debug breaking on ICEs on windows #145976

Open

Kobzol deleted the bootstrap-test branch September 1, 2025 06:44

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Enforce in bootstrap that test must have stage at least 1 (except for compiletest) #145663

Uh oh!

Conversation

Kobzol commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 20, 2025

Uh oh!

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Aug 20, 2025

Uh oh!

Kobzol commented Aug 20, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Aug 20, 2025

Uh oh!

This comment has been minimized.

rustbot commented Aug 30, 2025

Uh oh!

Kobzol commented Aug 30, 2025

Uh oh!

This comment has been minimized.

rust-bors bot commented Aug 30, 2025

Uh oh!

Kobzol commented Aug 30, 2025

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Sep 1, 2025

Uh oh!

bors commented Sep 1, 2025

Uh oh!

bors commented Sep 1, 2025

Uh oh!

bors commented Sep 1, 2025

Uh oh!

Uh oh!

github-actions bot commented Sep 1, 2025

Test differences

Stage 0

Stage 1

Stage 2

Job duration changes

Uh oh!

rust-timer commented Sep 1, 2025

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

Uh oh!

Kobzol commented Aug 20, 2025 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading