rustdoc: Test & document test_harness code block attribute

Author: fmease

src/doc/rustdoc/src/write-documentation/documentation-tests.md

@@ -321,21 +321,43 @@ we can add the `#[macro_use]` attribute. Second, we’ll need to add our own
 
 ## Attributes
 
-Code blocks can be annotated with attributes that help `rustdoc` do the right
-thing when testing your code:
+Code blocks can be annotated with attributes that tell `rustdoc` how to build and interpret the test.
+They follow the triple backtick in the opening line.
+As such, they share the place with language strings like `rust` or `text`.
+Multiple attributes can be provided by separating them with commas, spaces or tabs.
+You can also write comments which are enclosed in parentheses `(…)`.
 
-The `ignore` attribute tells Rust to ignore your code. This is almost never
-what you want as it's the most generic. Instead, consider annotating it
-with `text` if it's not code or using `#`s to get a working example that
-only shows the part you care about.
+As alluded to in the introduction at the very top,
+unless you specify `rust` or something that isn't an attribute (except for `custom`),
+the code block is assumed to be Rust source code (and is syntax highlighted as such).
+
+You can of course add `rust` explicitly (like `rust,ignore`) if the Markdown is also consumed by
+other tools (e.g., if it's contained inside of a `README.md` that's included via `include_str`).
+
+### `ignore`
+
+The `ignore` attribute tells `rustdoc` to ignore your code. This is useful if you would like to
+have Rust syntax highlighting but the snippet is incomplete or pseudocode.
+It is customary to add the reason why it should be ignored in a `(…)` comment.
 
 ```rust
 /// ```ignore
 /// fn foo() {
 /// ```
+///
+/// ```ignore (needs extra dependency)
+/// use dependency::functionality;
+/// functionality();
+/// ```
 # fn foo() {}
 ```
 
+Do note that this is almost never what you want as it's the most generic.
+Instead, consider annotating it with `text` if it's not code or
+using `#`s to get a working example that only shows the part you care about.
+
+### `should_panic`
+
 `should_panic` tells `rustdoc` that the code should compile correctly but
 panic during execution. If the code doesn't panic, the test will fail.
 
@@ -346,6 +368,8 @@ panic during execution. If the code doesn't panic, the test will fail.
 # fn foo() {}
 ```
 
+### `no_run`
+
 The `no_run` attribute will compile your code but not run it. This is
 important for examples such as "Here's how to retrieve a web page,"
 which you would want to ensure compiles, but might be run in a test
@@ -361,10 +385,10 @@ used to demonstrate code snippets that can cause Undefined Behavior.
 # fn foo() {}
 ```
 
+### `compile_fail`
+
 `compile_fail` tells `rustdoc` that the compilation should fail. If it
-compiles, then the test will fail. However, please note that code failing
-with the current Rust release may work in a future release, as new features
-are added.
+compiles, then the test will fail.
 
 ```rust
 /// ```compile_fail
@@ -374,6 +398,13 @@ are added.
 # fn foo() {}
 ```
 
+<div class="warning">
+However, please note that code failing with the current Rust release may work in a future release,
+as new features are added!
+</div>
+
+### `edition…`
+
 `edition2015`, `edition2018`, `edition2021`, and `edition2024` tell `rustdoc`
 that the code sample should be compiled using the respective edition of Rust.
 
@@ -390,6 +421,8 @@ that the code sample should be compiled using the respective edition of Rust.
 # fn foo() {}
 ```
 
+### `standalone_crate`
+
 Starting in the 2024 edition[^edition-note], compatible doctests are merged as one before being
 run. We combine doctests for performance reasons: the slowest part of doctests is to compile them.
 Merging all of them into one file and compiling this new file, then running the doctests is much
@@ -441,7 +474,7 @@ should not be merged with the others. So the previous code should use it:
 In this case, it means that the line information will not change if you add/remove other
 doctests.
 
-### Ignoring targets
+### `ignore-…`: Ignoring targets
 
 Attributes starting with `ignore-` can be used to ignore doctests for specific
 targets. For example, `ignore-x86_64` will avoid building doctests when the
@@ -478,7 +511,7 @@ struct Foo;
 In older versions, this will be ignored on all targets, but starting with
 version 1.88.0, `ignore-x86_64` will override `ignore`.
 
-### Custom CSS classes for code blocks
+### `{…}` & `custom`: Custom CSS classes for code blocks
 
 ```rust
 /// ```custom,{class=language-c}
@@ -504,8 +537,9 @@ To be noted that you can replace `class=` with `.` to achieve the same result:
 pub struct Bar;
 ```
 
-To be noted, `rust` and `.rust`/`class=rust` have different effects: `rust` indicates that this is
-a Rust code block whereas the two others add a "rust" CSS class on the code block.
+To be noted, `rust` and `{.rust}` / `{class=rust}` have different effects:
+`rust` indicates that this is a Rust code block whereas
+the two others add a "rust" CSS class on the code block.
 
 You can also use double quotes:
 
@@ -516,6 +550,27 @@ You can also use double quotes:
 pub struct Bar;
 ```
 
+### `test_harness`
+
+With `test_harness` applied, `rustdoc` will run any contained *test functions*
+instead of the (potentially implicit) `main` function.
+
+```rust
+//! ```test_harness
+//! #[test]
+//! #[should_panic]
+//! fn abc() { assert!(false); }
+//!
+//! #[test]
+//! fn xyz() { assert!(true); }
+//! ```
+```
+
+You can read more about *test functions* in [the Book][testing-book] or in [the Rust Reference][testing-ref].
+
+[testing-book]: ../../book/book/ch11-01-writing-tests.html
+[testing-ref]: ../../reference/attributes/testing.html
+
 ## Syntax reference
 
 The *exact* syntax for code blocks, including the edge cases, can be found

src/librustdoc/html/markdown.rs

@@ -847,11 +847,12 @@ pub(crate) enum Ignore {
     Some(Vec<String>),
 }
 
-/// This is the parser for fenced codeblocks attributes. It implements the following eBNF:
+/// This is the parser for fenced codeblocks attributes.
 ///
-/// ```eBNF
-/// lang-string = *(token-list / delimited-attribute-list / comment)
+/// It implements the following grammar as expressed in ABNF:
 ///
+/// ```ABNF
+/// lang-string = *(token-list / delimited-attribute-list / comment)
 /// bareword = LEADINGCHAR *(CHAR)
 /// bareword-without-leading-char = CHAR *(CHAR)
 /// quoted-string = QUOTE *(NONQUOTE) QUOTE
@@ -862,7 +863,7 @@ pub(crate) enum Ignore {
 /// attribute-list = [sep] attribute *(sep attribute) [sep]
 /// delimited-attribute-list = OPEN-CURLY-BRACKET attribute-list CLOSE-CURLY-BRACKET
 /// token-list = [sep] token *(sep token) [sep]
-/// comment = OPEN_PAREN *(all characters) CLOSE_PAREN
+/// comment = OPEN_PAREN *<all characters except closing parentheses> CLOSE_PAREN
 ///
 /// OPEN_PAREN = "("
 /// CLOSE_PARENT = ")"

tests/rustdoc-ui/doctest/test-harness.rs

@@ -0,0 +1,44 @@
+// Ensure that the code block attr `test_harness` works as expected.
+//@ compile-flags: --test --test-args --test-threads=1
+//@ normalize-stdout: "tests/rustdoc-ui/doctest" -> "$$DIR"
+//@ normalize-stdout: "finished in \d+\.\d+s" -> "finished in $$TIME"
+//@ rustc-env: RUST_BACKTRACE=0
+//@ failure-status: 101
+
+// The `main` fn won't be run under `test_harness`, so this test should pass.
+//! ```test_harness
+//! fn main() {
+//!     assert!(false);
+//! }
+//! ```
+
+// Check that we run both `bad` and `good` even if `bad` fails.
+//! ```test_harness
+//! #[test]
+//! fn bad() {
+//!     assert!(false);
+//! }
+//!
+//! #[test]
+//! fn good() {
+//!     assert!(true);
+//! }
+//! ```
+
+// Contrary to normal doctests, cfg `test` is active under `test_harness`.
+//! ```test_harness
+//! #[cfg(test)]
+//! mod group {
+//!     #[test]
+//!     fn element() {
+//!         assert!(false);
+//!     }
+//! }
+//! ```
+
+// `test_harness` doctests aren't implicitly wrapped in a `main` fn even if they contain stmts.
+//! ```test_harness
+//! assert!(true);
+//!
+//! #[test] fn extra() {}
+//! ```

tests/rustdoc-ui/doctest/test-harness.stdout

@@ -0,0 +1,76 @@
+
+running 4 tests
+test $DIR/test-harness.rs - (line 14) ... FAILED
+test $DIR/test-harness.rs - (line 25) ... FAILED
+test $DIR/test-harness.rs - (line 34) ... FAILED
+test $DIR/test-harness.rs - (line 9) ... ok
+
+failures:
+
+---- $DIR/test-harness.rs - (line 14) stdout ----
+Test executable failed (exit status: 101).
+
+stdout:
+
+running 2 tests
+test good ... ok
+test bad ... FAILED
+
+failures:
+
+---- bad stdout ----
+
+thread 'bad' ($TID) panicked at $DIR/test-harness.rs:4:5:
+assertion failed: false
+note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
+
+
+failures:
+    bad
+
+test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in $TIME
+
+
+
+---- $DIR/test-harness.rs - (line 25) stdout ----
+Test executable failed (exit status: 101).
+
+stdout:
+
+running 1 test
+test group::element ... FAILED
+
+failures:
+
+---- group::element stdout ----
+
+thread 'group::element' ($TID) panicked at $DIR/test-harness.rs:6:9:
+assertion failed: false
+note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
+
+
+failures:
+    group::element
+
+test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in $TIME
+
+
+
+---- $DIR/test-harness.rs - (line 34) stdout ----
+error: non-item macro in item position: assert
+  --> $DIR/test-harness.rs:35:1
+   |
+LL | assert!(true);
+   | ^^^^^^^^^^^^^
+
+error: aborting due to 1 previous error
+
+Couldn't compile the test.
+
+failures:
+    $DIR/test-harness.rs - (line 14)
+    $DIR/test-harness.rs - (line 25)
+    $DIR/test-harness.rs - (line 34)
+
+test result: FAILED. 1 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in $TIME
+