Flag for stop interpreting code blocks as doctests

[hghwng]

Code blocks inside documentation are interpreted as doctests by default. Sometimes the code is not for testing at all (for example, writing pseudo-code for an algorithm), and we can add ignore at the beginning of the code block.

However, even if ignore is added, cargo test still interprets the code block as doctest, and prints "... ignored" on each occurrence of ignore-tagged code blocks. In fact, those code blocks should not be interpreted as doctest, and cargo test should silently ignore those code blocks.

For example, the following code is a snippet of my current project.

/// Minimize the input size while preserving the coverage.
///
/// The algorithm trims by trying to remove chunks, starting from large chunks
/// to small chunks.
///
/// # Algorithm
/// ```ignore
/// for round in rounds {
///     for chunk_index in buffer {
///         let new_buffer = remove_chunk(chunk_index, round_size);
///         if run(new_buffer).coverage == known_coverage {
///             buffer = new_buffer;
///         }
///     }
/// }
/// ```
pub struct Trim {
...
}

cargo test generates the following output:

[...]
   Doc-tests dfuzz

running 4 tests
test src/algorithm/pass/mod.rs - algorithm::pass::Trim (line 17) ... ignored
test src/tracer/function.rs - tracer::function (line 18) ... ignored
test src/tracer/function.rs - tracer::function (line 23) ... ignored
test src/tracer/function.rs - tracer::function (line 5) ... ignored

Is it possible to add a flag in the code block to suppress such "ignored" messages?

[jonas-schievink]

You can already use something like notrust instead of ignore to get this effect. Not sure if this is documented anywhere though...

[Lonami]

Would it still get Rust's syntax highlight with notrust?

[jonas-schievink]

Ah, good point. I don't think so.

[ExpHP]

The issue with ignore is worse than just the fact that they get counted by the harness. If you do

cargo test -- --ignored

it runs them. Hence, ignore is only suitable for tests that take a long time to run, as opposed to snippets of code that should never be tested.

[jlgerber]

I was just googling to see about a solution to this very problem. I too have code that I don't want doctests run on for the very same reasons. I have never seen norust documented anywhere, and it is surely a step up from ignore , but the current solution still has a number of drawbacks as has been pointed out.
Perhaps cargo could
(a) ignore any code block with an unrecognized tag (so if you tag your codeblock with, say, yaml it would just ignore it)
(b) respect # ignore and/or # notest within the body of the code block, assuming it is rust

[shreevatsa]

For what it's worth, for a code block in the doc comment, instead of starting it with ignore or notrust I randomly put

    ```below-is-actually-rust-but-not-doctests-see-https://github.com/rust-lang/rust/issues/63193

and somehow it seems to "work" in the sense that cargo test -- --ignored does not pick it up, and cargo doc uses Rust syntax highlighting for that code block!

Some things I tried didn't work: obviously this is a horrible hack and probably happens to work for now because of some bug in rustdoc or whatever, but just sharing it here in case it's of interest to anyone :-)

[orenbenkiki]

Does it make sense to consider a feature request for an example indicator, which works similarly to ignore, but is not picked up by cargo test --ignored?

[detly]

@shreevatsa It seems the luck ran out on your trick there 🙂

[Sushisource]

It's pretty frustrating to have to choose between syntax highlighting and "this code is never to be compiled or tested regardless of flags" -- would be really nice to have a simple way to handle this seemingly common case

[Aloso]

I found a solution. It's a hack, but it works:

/// Awesome function!
///
#[cfg_attr(doctest, doc = " ````no_test")]
/// ```
/// // Code that is syntax-highlighted, but not tested!
/// ````
pub fn documented_function() {}

To understand how it works, you need to know the following:

• In Rust, /// comments are equivalent to a #[doc = ""] attribute
• In CommonMark, a fenced code block that starts with n backticks (n ≥ 3) can be closed with at least n backticks.

Line 3 inserts ````no_test, but only if doctests are being run. If that line is present, lines 3-6 are skipped because rustc sees the no_test attribute and assumes that this isn't Rust code. However, while documenting, line 3 is skipped and lines 4-6 are parsed as a Rust code block.

[CAD97]

Posting this here as well as #87586, since it's highly relevant to the OP here as well:

I think ```rust,ignore should behave identically to #[ignore]; that is, they should actually be checked to compile, but not run unless you pass --ignored. That this isn't checked to compile I personally think is a bug, though one I don't think we can fix.

My ideal state of the world:

doc codeblock tag	test attribute	compile	run
ignore	ignore	yes1	with --ignored
no_run	no_run2	yes	no
compile_fail		fail
no_compile3	cfg(any())	no

Given current usage of ```ignore, though, I think the best we can reasonably ask for is something along the lines of

doc codeblock tag	test attribute	compile	run
maybe_ignore4	ignore	yes	with --ignored
no_run	no_run	yes	no
compile_fail	n/a	fail
ignore	cfg(any(())	no

Footnotes

1. currently doctests only compile check with --ignored ↩

2. polyfill is #[cfg(test)] const MY_TEST: fn() = || { .. } instead of #[test] fn my_test() { .. } ↩

3. polyfill is #[cfg_attr(doctest, doc = " ````text"] ↩

4. modulo bikeshed: picking a name here is hard since it can't be #[ignore] ↩

[cospectrum]

#[cfg(not(doctest))] helped in my case, but it will be applied to all code in scope

/// Broken code:
/// ```
/// panic!("...");
/// compile_error!("...");
/// ```
#[cfg(not(doctest))]
pub fn f() {
    // ...
}

[benfrankel]

This issue is super annoying. Here's a workaround I'm using:

/// This doctest passes:
///
/// ```
/// # /*
/// x += y;
/// # */
/// ```

This interferes with syntax highlighting in my IDE, but the rendered docs are fine.