Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustdoc runs some markdown tests twice #42726

Closed
ollie27 opened this issue Jun 17, 2017 · 1 comment
Closed

rustdoc runs some markdown tests twice #42726

ollie27 opened this issue Jun 17, 2017 · 1 comment
Labels
regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-dev-tools Relevant to the dev-tools subteam, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@ollie27
Copy link
Member

ollie27 commented Jun 17, 2017

Running rustdoc --test with 1.18.0 and nightly on the following markdown file:

# Foo

```
fail1
```

```
fail2
```

bar
```
fail3
```

Produces the following:

output
running 5 tests
test foo.md - Foo (line 3) ... FAILED
test foo.md - Foo (line 7) ... FAILED
test foo.md - Foo (line 12) ... FAILED
test foo.md - Foo (line 8) ... FAILED
test foo.md - Foo (line 3) ... FAILED

failures:

---- foo.md - Foo (line 3) stdout ----
        error[E0425]: cannot find value `fail1` in this scope
 --> <anon>:2:1
  |
2 | fail1
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218
note: Run with `RUST_BACKTRACE=1` for a backtrace.

---- foo.md - Foo (line 7) stdout ----
        error[E0425]: cannot find value `fail2` in this scope
 --> <anon>:2:1
  |
2 | fail2
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218

---- foo.md - Foo (line 12) stdout ----
        error[E0425]: cannot find value `fail3` in this scope
 --> <anon>:2:1
  |
2 | fail3
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218

---- foo.md - Foo (line 8) stdout ----
        error[E0425]: cannot find value `fail2` in this scope
 --> <anon>:2:1
  |
2 | fail2
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218

---- foo.md - Foo (line 3) stdout ----
        error[E0425]: cannot find value `fail1` in this scope
 --> <anon>:2:1
  |
2 | fail1
  | ^^^^^ not found in this scope

error: aborting due to previous error(s)

thread 'rustc' panicked at 'Box<Any>', src\librustc\session\mod.rs:218


failures:
    foo.md - Foo (line 12)
    foo.md - Foo (line 3)
    foo.md - Foo (line 3)
    foo.md - Foo (line 7)
    foo.md - Foo (line 8)

test result: FAILED. 0 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out

The first 2 tests are run twice (one with the wrong line number though). The last test is only run once. It looks as though it's running all the tests found by hoedown and all the tests run by pulldown-cmark. I'm pretty sure this wasn't the intention of #41290 and #41431 cc. @GuillaumeGomez. Note that this doesn't happen for tests in rust files.

This is also technically a stable to stable regression because the following file passes rustdoc --test on 1.17.0 but not on 1.18.0 or later:

# Foo

bar
```
fail3
```
@GuillaumeGomez
Copy link
Member

Which is strange since I didn't change the way doc tests are run in any version. :-/

I'll take a look when I'm back.

@steveklabnik steveklabnik added T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. T-dev-tools Relevant to the dev-tools subteam, which will review and decide on the PR/issue. labels Jun 19, 2017
@Mark-Simulacrum Mark-Simulacrum added the regression-from-stable-to-stable Performance or correctness regression from one stable version to another. label Jul 5, 2017
bors added a commit that referenced this issue Jul 7, 2017
rustdoc: Don't run Markdown tests twice

This matches the behaviour for finding tests in Rust files.

This was a regression from 1.17 to 1.18 so it would be a good idea to backport this to beta so at least 1.19 won't also be affected.

Fixes #42726

r? @GuillaumeGomez
jhecking added a commit to aerospike/aerospike-client-rust that referenced this issue Jan 25, 2018
rustdoc --test with 1.18.0 is running tests twice, which leads to a race
condition where the second test run might fail if the first test run
deletes the shared record. See rust-lang/rust#42726.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-dev-tools Relevant to the dev-tools subteam, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

4 participants