rustdoc: refactor how passes are structured, and turn intra-doc-link collection into a pass #52800

QuietMisdreavus · 2018-07-28T04:26:10Z

This builds on #52751 and should not be merged until that finally finishes the bors queue

Part 2 of my passes refactor. This introduces the concept of an "early pass", which is run right before exiting the compiler context. This is important for passes that may want to ask the compiler about things. For example, i took out the intra-doc-link collection and turned it into a early pass. I also made the strip-hidden, strip-private and strip-priv-imports passes occur as early passes, so that intra-doc-link collection wouldn't run on docs that weren't getting printed.

Fixes #51684, technically #51468 too but that version of h2 hits a legit intra-link error after that >_>

r? @rust-lang/rustdoc

QuietMisdreavus · 2018-07-28T04:47:18Z

An analysis of the bug that caused cfg-if and bytes to break a bunch of crates once we turned resolution errors into a proper lint:

cfg-if v0.1.3 has the following at the top of its crate docs:

//! A macro for defining #[cfg] if-else statements.

bytes v0.4.8 has the following in its crate docs:

//! A `Bytes` handle can be created directly from an existing byte store (such as &[u8]
//! or Vec<u8>), but usually a `BytesMut` is used first and written to. For
//! example:

When rustdoc processes a crate, it currently processes intra-doc-links as soon as the item is "cleaned". This is fairly early in the process, and occurs before any decision of whether to show a given item's documentation has been made. (Sure, we have a sense of whether it's in a public module, but not whether it's been exported somewhere that is public.) Furthermore, whenever it finds a problem resolving a link, it reports an error immediately, again ignoring whether that item is relevant for the current doc run.

Generally, whenever we encounter an external item, we only import its definition and attributes once we know we're going to display it. Either it's being used in a pub use in a public module, or someone put a #[doc(inline)] on it. However, for extern crate statements, we process their attributes just like it were a struct or function. This is a problem only because rustc loads all the inner attributes of a crate and throws them in alongside the attributes tagged onto the extern crate statement. Including their docs.

Which brings us back to cfg-if and bytes. Whenever some project used either of those crates, those links would be treated as local (despite not being written locally) and the error would be reported. Combine this with crates that use #![deny(warnings)] and you have the error that brought down h2 and a few more crates.

My original reading of the issue is still valid, though, even if it's not the bug that created the issue. The other test i added also demonstrates this. A private item with a bad link target on it will currently report an error, despite not appearing in the documentation to create a broken link.

GuillaumeGomez · 2018-07-28T15:21:19Z

Looks good to me! Please squash your tidy commit though. ;)

QuietMisdreavus · 2018-07-29T16:29:29Z

Now that #52751 is landed, i've rebased atop master so that commit doesn't show up in this one.

GuillaumeGomez · 2018-07-29T21:16:57Z

Then let's get it in!

@bors: r+

bors · 2018-07-29T21:16:58Z

📌 Commit 85bf440f98cd634563d1d9a1e24cd5b6027917fe has been approved by GuillaumeGomez

bors · 2018-07-29T23:48:05Z

🔒 Merge conflict

This pull request and the master branch diverged in a way that cannot be automatically merged. Please rebase on top of the latest master branch, and let the reviewer approve again.

How do I rebase?

Assuming self is your fork and upstream is this repository, you can resolve the conflict following these steps:

git checkout do-not-pass-go (switch to your branch)
git fetch upstream master (retrieve the latest master)
git rebase upstream/master -p (rebase on top of it)
Follow the on-screen instruction to resolve conflicts (check git status if you got lost).
git push self do-not-pass-go --force-with-lease (update this PR)

You may also read Git Rebasing to Resolve Conflicts by Drew Blessing for a short tutorial.

Please avoid the "Resolve conflicts" button on GitHub. It uses git merge instead of git rebase which makes the PR commit history more difficult to read.

Sometimes step 4 will complete without asking for resolution. This is usually due to difference between how Cargo.lock conflict is handled during merge and rebase. This is normal, and you should still perform step 5 to update this PR.

Error message

Auto-merging src/librustdoc/lib.rs
CONFLICT (content): Merge conflict in src/librustdoc/lib.rs
Auto-merging src/librustdoc/core.rs
Automatic merge failed; fix conflicts and then commit the result.

kennytm · 2018-07-30T23:23:58Z

@bors r-

Wrongly rescheduled.

QuietMisdreavus · 2018-07-31T16:16:14Z

Rebased.

QuietMisdreavus · 2018-08-03T15:31:51Z

@bors p=1

It seems that any time someone touches the current intra-doc links functionality, this PR gets a merge conflict because of how it's moving the code around. I just had to rebase again.

QuietMisdreavus · 2018-08-03T17:59:05Z

@bors r=GuillaumeGomez

bors · 2018-08-03T17:59:06Z

📌 Commit e47dea72472b2f2f4b978b72b95b63edbb1d93e8 has been approved by GuillaumeGomez

bors · 2018-08-03T19:49:19Z

⌛ Testing commit e47dea72472b2f2f4b978b72b95b63edbb1d93e8 with merge 1580002f9fa813f468c31d391b00d12399b8507b...

bors · 2018-08-03T21:10:56Z

💔 Test failed - status-appveyor

QuietMisdreavus · 2018-08-03T21:28:12Z

------------------------------------------
stderr:
------------------------------------------
error: Unrecognized option: 'invalid-arg-foo'
error: kaboom
  --> compile-error.rs:12:5
   |
12 |     compile_error!("kaboom");
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^
error: aborting due to previous error
error: kaboom
  --> compile-error.rs:12:5
   |
12 |     compile_error!("kaboom");
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^
thread 'main' panicked at 'encountered error with `-Z treat_err_as_bug', librustc_errors\lib.rs:478:13
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: rustc 1.30.0-dev running on x86_64-pc-windows-msvc
note: compiler flags: -Z treat-err-as-bug
error: Unrecognized option: 'invalid-arg-foo'
error: kaboom
  --> compile-error.rs:12:5
   |
12 |     compile_error!("kaboom");
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^
make: *** [Makefile:11: all] Error 1
------------------------------------------
thread '[run-make] run-make-fulldeps\exit-code' panicked at 'explicit panic', tools\compiletest\src\runtest.rs:3149:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.
failures:
    [run-make] run-make-fulldeps\exit-code
test result: FAILED. 187 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out

...did i mess with rustdoc's exit code somehow? was that modified in the last week and i somehow missed it in my rebases?

EDIT: Wait, that's rustc's exit code. It's the only one being run with -Z treat-err-as-bug, and it's claiming to be rustc anyway. This PR doesn't touch anything outside librustdoc; what's going on? I'm reading this wrong, it's the stderr of every run in that makefile.

QuietMisdreavus · 2018-08-03T21:50:48Z

Ohhhh, i see it now. The run-make-fulldeps/exit-code test tries to document a crate with this, so that it will exit with an error:

#![deny(intra_doc_link_resolution_failure)]

/// [intradoc::failure]
fn main() {
    println!("Hello, world!");
}

However, now that we're not running docs for private items, the lint that was being emitted there is no longer being emitted. I'll update the test.

rust-highfive · 2018-08-05T01:02:32Z

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2018-08-05T04:28:47Z

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:43:49] ....................................................................................................
[00:43:51] ....................................................................................................
[00:43:54] ....................................................................................................
[00:43:57] ....................................................................................................
[00:43:59] iiiiiiiii...........................................................................................
[00:44:05] ....................................................................................................
[00:44:09] .....i..............................................................................................
[00:44:11] .........i..........................................................................................
[00:44:14] ....................................................................................................

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

QuietMisdreavus · 2018-08-05T14:33:13Z

Broadcast message from root@travis-job-d3ffdbb6-fb99-47ea-a99b-4def481baca7

	(unknown) at 4:28 ...




The system is going down for power off NOW!

[01:06:20] 
[01:06:20] Session terminated, terminating shell...make: *** wait: No child processes.  Stop.
[01:06:20] make: *** Waiting for unfinished jobs....
[01:06:20] make: *** wait: No child processes.  Stop.
[01:06:20]  ...terminated.

I'm restarting the travis build.

EDIT: I've been told travis people are looking at it, so i'll link their issue here so it's saved. travis-ci/travis-ci#4924

QuietMisdreavus · 2018-08-05T16:00:42Z

Travis is green, so let's get this rolling again...

@bors r=GuillaumeGomez

bors · 2018-08-05T16:00:43Z

📌 Commit e332985 has been approved by GuillaumeGomez

bors · 2018-08-05T18:45:08Z

⌛ Testing commit e332985 with merge 73c7873...

rustdoc: refactor how passes are structured, and turn intra-doc-link collection into a pass This builds on #52751 and should not be merged until that finally finishes the bors queue Part 2 of my passes refactor. This introduces the concept of an "early pass", which is run right before exiting the compiler context. This is important for passes that may want to ask the compiler about things. For example, i took out the intra-doc-link collection and turned it into a early pass. I also made the `strip-hidden`, `strip-private` and `strip-priv-imports` passes occur as early passes, so that intra-doc-link collection wouldn't run on docs that weren't getting printed. Fixes #51684, technically #51468 too but that version of `h2` hits a legit intra-link error after that `>_>` r? @rust-lang/rustdoc

bors · 2018-08-05T20:45:00Z

☀️ Test successful - status-appveyor, status-travis
Approved by: GuillaumeGomez
Pushing 73c7873 to master...

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 28, 2018

This comment has been minimized.

Sign in to view

QuietMisdreavus mentioned this pull request Jul 28, 2018

rustdoc: only show intra_doc_link_resolution_failure lints for items being documented #51684

Closed

QuietMisdreavus force-pushed the do-not-pass-go branch from a6d5075 to 08c1e72 Compare July 28, 2018 05:08

QuietMisdreavus force-pushed the do-not-pass-go branch 2 times, most recently from 260c641 to 85bf440 Compare July 29, 2018 16:28

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 29, 2018

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 29, 2018

Mark-Simulacrum mentioned this pull request Jul 30, 2018

Rollup of 22 pull requests #52882

Closed

QuietMisdreavus force-pushed the do-not-pass-go branch from 85bf440 to f3c7ea4 Compare July 31, 2018 15:18

QuietMisdreavus added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 1, 2018

QuietMisdreavus force-pushed the do-not-pass-go branch from f3c7ea4 to e47dea7 Compare August 3, 2018 15:24

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 3, 2018

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 3, 2018

QuietMisdreavus added 2 commits August 4, 2018 18:44

strongly-typed passes

aaec101

introduce "early passes" an convert a few over

d3d6f76

QuietMisdreavus force-pushed the do-not-pass-go branch from e16a2e4 to bb4f588 Compare August 5, 2018 00:21

QuietMisdreavus added 5 commits August 4, 2018 22:20

turn intra-doc-link collection into an early pass

9e4e266

add tests for new intra-doc-link behavior

001445b

don't process intra-links on extern-crate items

bef0f25

only abort after all early passes have run

323b2b7

fix exit-code test so the lint fires again

e332985

QuietMisdreavus force-pushed the do-not-pass-go branch from bb4f588 to e332985 Compare August 5, 2018 03:20

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 5, 2018

bors merged commit e332985 into rust-lang:master Aug 5, 2018

QuietMisdreavus deleted the do-not-pass-go branch February 12, 2020 00:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rustdoc: refactor how passes are structured, and turn intra-doc-link collection into a pass #52800

rustdoc: refactor how passes are structured, and turn intra-doc-link collection into a pass #52800

QuietMisdreavus commented Jul 28, 2018

This comment has been minimized.

QuietMisdreavus commented Jul 28, 2018

GuillaumeGomez commented Jul 28, 2018

QuietMisdreavus commented Jul 29, 2018

GuillaumeGomez commented Jul 29, 2018

bors commented Jul 29, 2018

bors commented Jul 29, 2018

kennytm commented Jul 30, 2018

QuietMisdreavus commented Jul 31, 2018

QuietMisdreavus commented Aug 3, 2018

QuietMisdreavus commented Aug 3, 2018

bors commented Aug 3, 2018

bors commented Aug 3, 2018

bors commented Aug 3, 2018

QuietMisdreavus commented Aug 3, 2018 •

edited

Loading

QuietMisdreavus commented Aug 3, 2018

rust-highfive commented Aug 5, 2018

rust-highfive commented Aug 5, 2018

QuietMisdreavus commented Aug 5, 2018 •

edited

Loading

QuietMisdreavus commented Aug 5, 2018

bors commented Aug 5, 2018

bors commented Aug 5, 2018

bors commented Aug 5, 2018

rustdoc: refactor how passes are structured, and turn intra-doc-link collection into a pass #52800

rustdoc: refactor how passes are structured, and turn intra-doc-link collection into a pass #52800

Conversation

QuietMisdreavus commented Jul 28, 2018

This comment has been minimized.

QuietMisdreavus commented Jul 28, 2018

GuillaumeGomez commented Jul 28, 2018

QuietMisdreavus commented Jul 29, 2018

GuillaumeGomez commented Jul 29, 2018

bors commented Jul 29, 2018

bors commented Jul 29, 2018

kennytm commented Jul 30, 2018

QuietMisdreavus commented Jul 31, 2018

QuietMisdreavus commented Aug 3, 2018

QuietMisdreavus commented Aug 3, 2018

bors commented Aug 3, 2018

bors commented Aug 3, 2018

bors commented Aug 3, 2018

QuietMisdreavus commented Aug 3, 2018 • edited Loading

QuietMisdreavus commented Aug 3, 2018

rust-highfive commented Aug 5, 2018

rust-highfive commented Aug 5, 2018

QuietMisdreavus commented Aug 5, 2018 • edited Loading

QuietMisdreavus commented Aug 5, 2018

bors commented Aug 5, 2018

bors commented Aug 5, 2018

bors commented Aug 5, 2018

QuietMisdreavus commented Aug 3, 2018 •

edited

Loading

QuietMisdreavus commented Aug 5, 2018 •

edited

Loading