rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag #149565

jyn514 · 2025-12-02T21:59:41Z

This is useful for changing the default for whether doctests are merged or not. Currently, that default is solely controlled by edition = 2024, which adds a high switching cost to get doctest merging. This flag allows opting in even on earlier editions.

Unlike the edition = 2024 default, --merge-doctests=yes gives a hard error if merging fails instead of falling back to running standalone tests. The user has explicitly said they want merging, so we shouldn't silently do something else.

--merge-doctests=auto is equivalent to the current 2024 edition behavior, but available on earlier editions.

Helps with #141240. @epage said in that issue he would like a per-doctest opt-in, and that seems useful to me in addition to this flag, but I think it's a separate use case from changing the default.

rustbot · 2025-12-02T21:59:47Z

r? @notriddle

rustbot has assigned @notriddle.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

src/librustdoc/doctest.rs

jyn514 · 2025-12-02T22:27:05Z

tests/rustdoc-ui/doctest/force-merge-fail.stderr

+doctest_bundle_2018.rs:$LINE:$COL: error: `unmangled_name` redeclared with a different signature: this signature doesn't match the previous declaration
+error: aborting due to 1 previous error
+error: failed to merge doctests
+   |
+   = note: requested explicitly on the command line with `--merge-doctests=yes`


this error kinda sucks, but this is an unstable feature anyway, and with -C save-temps people will actually be able to inspect the merged code, which should make this easier to debug.

oh, maybe this should recommend -C save-temps as a note?

jyn514 · 2025-12-03T01:37:55Z

hm this broke the reference’s doctest somehow

      failures:
      
      ---- items/constant-items.md - Constant_items (line 71) stdout ----
      Test compiled successfully, but it's marked `compile_fail`.
      ---- items/constant-items.md - Constant_items (line 97) stdout ----
      Test compiled successfully, but it's marked `compile_fail`.
      
      failures:
          items/constant-items.md - Constant_items (line 71)
          items/constant-items.md - Constant_items (line 97)

jyn514 · 2025-12-03T17:38:30Z

i'm completely unable to reproduce this :/ i see ci has this output which i don't see locally:

[2025-12-03T00:03:32Z WARN  mdbook::preprocess::cmd] The command wasn't found, is the "spec" preprocessor installed?
[2025-12-03T00:03:32Z WARN  mdbook::preprocess::cmd] 	Command: cargo run --release --manifest-path mdbook-spec/Cargo.toml

jyn514 · 2025-12-03T17:43:07Z

i ran rustdoc +stage2 --test src/items/constant-items.md manually and it doesn't even show the same line numbers as CI :/ what on earth

rustbot · 2025-12-03T17:46:05Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

src/librustdoc/doctest/make.rs

jyn514 · 2025-12-03T20:01:03Z

something really weird is going on with compiletest; locally it applies the normalize-stdout directive, but in CI it ignores it and replaces $DIR. the two don't match, hence the failure, but I don't understand why $DIR would possibly work in CI but not locally.

for now I'm just going to change the normalization to match compiletest's normalization.

jyn514 · 2025-12-03T20:01:56Z

oh, i wonder if this is about absolute vs relative paths :/

jyn514 · 2025-12-03T20:19:50Z

i think the proper fix here is that rustdoc should use to_embeddable_absolute_path when -Z ui-testing is set. but i've already spent a bunch of time on this so i'm just going to open an issue and hopefully someone gets to it later.

This is useful for changing the *default* for whether doctests are merged or not. Currently, that default is solely controlled by `edition = 2024`, which adds a high switching cost to get doctest merging. This flag allows opt-ing in even on earlier additions. Unlike the `edition = 2024` default, `--merge-doctests=yes` gives a hard error if merging fails instead of falling back to running standalone tests. The user has explicitly said they want merging, so we shouldn't silently do something else. `--merge-doctests=auto` is equivalent to the current 2024 edition behavior, but available on earlier editions.

This allows viewing failed merged doctests.

epage · 2025-12-03T22:43:50Z

src/librustdoc/doctest/make.rs

-                && crate_attrs.is_empty()
+            // We try to look at the contents of the test to detect whether it should be merged.
+            // This is not a complete list of possible failures, but it catches many cases.
+            let will_probably_fail = has_global_allocator


Don't know why I'm getting a chuckle at that variable name :)

notriddle

This makes sense, and the code looks fine.

View changes since this review

jyn514 · 2025-12-08T15:23:56Z

reminder that i no longer have bors privileges and need someone else to approve :)

notriddle · 2025-12-11T16:32:43Z

@bors r+

bors · 2025-12-11T16:32:46Z

📌 Commit 415953a has been approved by notriddle

It is now in the queue for this repository.

…iddle rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag This is useful for changing the *default* for whether doctests are merged or not. Currently, that default is solely controlled by `edition = 2024`, which adds a high switching cost to get doctest merging. This flag allows opting in even on earlier editions. Unlike the `edition = 2024` default, `--merge-doctests=yes` gives a hard error if merging fails instead of falling back to running standalone tests. The user has explicitly said they want merging, so we shouldn't silently do something else. `--merge-doctests=auto` is equivalent to the current 2024 edition behavior, but available on earlier editions. Helps with rust-lang#141240. `@epage` said in that issue he would like a per-doctest opt-in, and that seems useful to me in addition to this flag, but I think it's a separate use case from changing the default.

…uwer Rollup of 10 pull requests Successful merges: - #142380 (Put negative implementors first and apply same ordering logic to foreign implementors) - #146584 (remove duplicated columns from `rustc_error_code::error_codes!`) - #148717 (Point at span within local macros even when error happens in nested external macro) - #148825 (Add SystemTime::{MIN, MAX}) - #149565 (rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag) - #149770 (Rename some issue-* tests) - #149807 (Use ubuntu:24.04 for the `x86_64-gnu-miri` job) - #149850 (Remove "tidy" tool for `tests/rustdoc` testsuite) - #149863 (Do not suggest moving expression out of for loop when hitting `break` from desugaring) - #149867 (only resolve main in bin crates) r? `@ghost` `@rustbot` modify labels: rollup

Rollup of 9 pull requests Successful merges: - #142380 (Put negative implementors first and apply same ordering logic to foreign implementors) - #146584 (remove duplicated columns from `rustc_error_code::error_codes!`) - #148717 (Point at span within local macros even when error happens in nested external macro) - #149565 (rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag) - #149770 (Rename some issue-* tests) - #149807 (Use ubuntu:24.04 for the `x86_64-gnu-miri` job) - #149850 (Remove "tidy" tool for `tests/rustdoc` testsuite) - #149863 (Do not suggest moving expression out of for loop when hitting `break` from desugaring) - #149867 (only resolve main in bin crates) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of #149565 - jyn514:force-doctest-merge, r=notriddle rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag This is useful for changing the *default* for whether doctests are merged or not. Currently, that default is solely controlled by `edition = 2024`, which adds a high switching cost to get doctest merging. This flag allows opting in even on earlier editions. Unlike the `edition = 2024` default, `--merge-doctests=yes` gives a hard error if merging fails instead of falling back to running standalone tests. The user has explicitly said they want merging, so we shouldn't silently do something else. `--merge-doctests=auto` is equivalent to the current 2024 edition behavior, but available on earlier editions. Helps with #141240. ``@epage`` said in that issue he would like a per-doctest opt-in, and that seems useful to me in addition to this flag, but I think it's a separate use case from changing the default.

Rollup of 9 pull requests Successful merges: - rust-lang/rust#142380 (Put negative implementors first and apply same ordering logic to foreign implementors) - rust-lang/rust#146584 (remove duplicated columns from `rustc_error_code::error_codes!`) - rust-lang/rust#148717 (Point at span within local macros even when error happens in nested external macro) - rust-lang/rust#149565 (rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag) - rust-lang/rust#149770 (Rename some issue-* tests) - rust-lang/rust#149807 (Use ubuntu:24.04 for the `x86_64-gnu-miri` job) - rust-lang/rust#149850 (Remove "tidy" tool for `tests/rustdoc` testsuite) - rust-lang/rust#149863 (Do not suggest moving expression out of for loop when hitting `break` from desugaring) - rust-lang/rust#149867 (only resolve main in bin crates) r? `@ghost` `@rustbot` modify labels: rollup

rustbot assigned notriddle Dec 2, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Dec 2, 2025

This comment was marked as resolved.

Sign in to view

This comment has been minimized.

Sign in to view

jyn514 changed the title ~~rustdoc: Add unstable --merge-doctests=yes/no flag~~ rustdoc: Add unstable --merge-doctests=yes/no/auto flag Dec 2, 2025

jyn514 force-pushed the force-doctest-merge branch 2 times, most recently from dde2ec7 to 0ff6d88 Compare December 2, 2025 22:23

jyn514 commented Dec 2, 2025

View reviewed changes

src/librustdoc/doctest.rs Show resolved Hide resolved

jyn514 commented Dec 2, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

rustbot added the A-run-make Area: port run-make Makefiles to rmake.rs label Dec 2, 2025

This comment has been minimized.

Sign in to view

jyn514 force-pushed the force-doctest-merge branch from b1e2ca9 to e2aa3fc Compare December 3, 2025 17:46

epage reviewed Dec 3, 2025

View reviewed changes

src/librustdoc/doctest/make.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

jyn514 mentioned this pull request Dec 3, 2025

rustdoc doctest UI suite shouldn't need to manually normalize $DIR #149616

Open

jyn514 added 2 commits December 3, 2025 15:48

Support -C save-temps in rustdoc

3aa4ece

This allows viewing failed merged doctests.

jyn514 mentioned this pull request Dec 3, 2025

Allow running x test reference locally #149617

Closed

jyn514 force-pushed the force-doctest-merge branch from 4a30a64 to 646667a Compare December 3, 2025 22:41

--merge-doctests is a default, not an override

415953a

jyn514 force-pushed the force-doctest-merge branch from 646667a to 415953a Compare December 3, 2025 22:43

epage reviewed Dec 3, 2025

View reviewed changes

notriddle approved these changes Dec 6, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 11, 2025

JonathanBrouwer mentioned this pull request Dec 11, 2025

Rollup of 10 pull requests #149890

Closed

matthiaskrgr mentioned this pull request Dec 11, 2025

Rollup of 9 pull requests #149891

Merged

bors merged commit 0dfc238 into rust-lang:main Dec 12, 2025
11 checks passed

rustbot added this to the 1.94.0 milestone Dec 12, 2025

jyn514 deleted the force-doctest-merge branch December 12, 2025 20:21

Uh oh!

rustdoc: Add unstable --merge-doctests=yes/no/auto flag #149565

rustdoc: Add unstable --merge-doctests=yes/no/auto flag #149565

Conversation

jyn514 commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Dec 2, 2025

Uh oh!

This comment was marked as resolved.

This comment has been minimized.

Uh oh!

jyn514 Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

jyn514 commented Dec 3, 2025

Uh oh!

jyn514 commented Dec 3, 2025

Uh oh!

jyn514 commented Dec 3, 2025

Uh oh!

rustbot commented Dec 3, 2025

Uh oh!

Uh oh!

This comment has been minimized.

jyn514 commented Dec 3, 2025

Uh oh!

jyn514 commented Dec 3, 2025

Uh oh!

jyn514 commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

epage Dec 3, 2025

Choose a reason for hiding this comment

Uh oh!

notriddle left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jyn514 commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

notriddle commented Dec 11, 2025

Uh oh!

bors commented Dec 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag #149565

rustdoc: Add unstable `--merge-doctests=yes/no/auto` flag #149565

jyn514 commented Dec 2, 2025 •

edited

Loading

jyn514 commented Dec 3, 2025 •

edited

Loading

notriddle left a comment •

edited by rustbot

Loading

jyn514 commented Dec 8, 2025 •

edited

Loading