rustdoc: overhaul code block lexing errors #56884
Conversation
|
(rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
| /// \_ | ||
| /// ``` | ||
| pub fn ok() {} | ||
|
|
We could do everything in just one pass, not sure we'd like that. For instance, we check intra links and if private items have doc tests in one pass. But maybe that'd be a nice thing to only have one pass and iterate over passes for each item. To be determined...
But at least you get the information, which is the most important in my opinion. |
|
Reading through now. This PR will probably take some time to give a real review for, though...
I wonder if we could make a kind of "item pass" so we could combine all the ones that need to check things, but not add new items, into one actual "pass" that runs multiple functions over each item as it finds it? That way we reduce the number of crate scans that need to happen, but still make the checks somewhat modular. |
|
@QuietMisdreavus I'm unclear on the scope of the changes that you'd like me to make here. Would it suffice to just attempt to combine the passes for gathering doctests and checking their syntax? |
|
Sorry, i was speaking hypothetically when i mentioned "item passes". This PR can stay using a separate pass to perform syntax-checking; the refactoring can come later. I'm reading over the PR now... |
euclio
force-pushed
the
codeblock-diagnostics
branch
from
431f58b
to
8c93798
Compare
|
Thanks so much! This is ready to go! @bors r+ |
|
📌 Commit 8c93798 has been approved by |
bors
added
S-waiting-on-bors
…etMisdreavus rustdoc: overhaul code block lexing errors Fixes rust-lang#53919. This PR moves the reporting of code block lexing errors from rendering time to an early pass, so we can use the compiler's error reporting mechanisms. This dramatically improves the diagnostics in this situation: we now de-emphasize the lexing errors as a note under a warning that has a span and suggestion instead of just emitting errors at the top level. Additionally, this PR generalizes the markdown -> source span calculation function, which should allow other rustdoc warnings to use better spans in the future. Last, the PR makes sure that the code block is always emitted in the docs, even if it fails to highlight correctly. Of note: - The new pass unfortunately adds another pass over the docs to gather the doc blocks for syntax-checking. I wonder if this could be combined with the pass that looks for testable blocks? I'm not familiar with that code, so I don't know how feasible that is. - `pulldown_cmark` doesn't make it easy to find the spans of the code blocks, so the code that calculates the spans is a little nasty. It works for all the test cases I threw at it, but I wouldn't be surprised if an edge case would break it. Should have a thorough review. - This PR worsens the state of rust-lang#56885, since those certain fatal lexing errors are now emitted before docs get generated at all.
Rollup of 6 pull requests Successful merges: - #56884 (rustdoc: overhaul code block lexing errors) - #57065 (Optimize try_mark_green and eliminate the lock on dep node colors) - #57107 (Add a regression test for mutating a non-mut #[thread_local]) - #57268 (Add a target option "merge-functions", and a corresponding -Z flag (works around #57356)) - #57551 (resolve: Add a test for issue #57539) - #57598 (Add missing unpretty option help message) Failed merges: r? @ghost
relax rustdoc output assertion The output of this rustdoc error is changing in rust-lang/rust#56884. This PR relaxes an assertion on the output so that the test will still pass once the rustdoc PR is merged.
|
⌛ Testing commit 8c93798 with merge 93780eacc1d356c8b8c4a5768fff26ff78b39e46... |
|
💔 Test failed - checks-travis |
bors
added
S-waiting-on-review
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
This looks like one of Cargo's tests, where it's scanning the output of rustdoc to make sure Cargo hasn't swallowed it. However, i'm not totally sure what the error is saying here. @rust-lang/cargo, this PR changes the output format of doctest failures, and it seems to have tripped up one of Cargo's tests. How should we change the Cargo test to make it work? (And how should we coordinate the submodule update with this PR? |
|
@QuietMisdreavus already fixed in rust-lang/cargo#6559. You'll have to rebase on top of #57721 to unblock this. |
|
Yeah, I noticed that failure yesterday. My fix is already in cargo master, we just have to wait for #57721 to land. |
|
Oh! Well, i'm glad it's almost already taken care of! 😅 |
Update cargo Unblocks #56884 cc @euclio 6 commits in 2b4a5f1f0bb6e13759e88ea9512527b0beba154f..ffe65875fd05018599ad07e7389e99050c7915be 2019-01-12 04:13:12 +0000 to 2019-01-17 23:57:50 +0000 - Better error message for bad manifest with `cargo install`. (rust-lang/cargo#6560) - relax rustdoc output assertion (rust-lang/cargo#6559) - touch some files when we use them (rust-lang/cargo#6477) - Add documentation for new package/publish feature flags. (rust-lang/cargo#6553) - Update chat link to Discord. (rust-lang/cargo#6554) - Fix typo (rust-lang/cargo#6552) r? @alexcrichton
|
This just needs a retry, then! |
|
On the assumption that this will Just Work... @bors retry |
bors
added
S-waiting-on-bors
rustdoc: overhaul code block lexing errors Fixes #53919. This PR moves the reporting of code block lexing errors from rendering time to an early pass, so we can use the compiler's error reporting mechanisms. This dramatically improves the diagnostics in this situation: we now de-emphasize the lexing errors as a note under a warning that has a span and suggestion instead of just emitting errors at the top level. Additionally, this PR generalizes the markdown -> source span calculation function, which should allow other rustdoc warnings to use better spans in the future. Last, the PR makes sure that the code block is always emitted in the docs, even if it fails to highlight correctly. Of note: - The new pass unfortunately adds another pass over the docs to gather the doc blocks for syntax-checking. I wonder if this could be combined with the pass that looks for testable blocks? I'm not familiar with that code, so I don't know how feasible that is. - `pulldown_cmark` doesn't make it easy to find the spans of the code blocks, so the code that calculates the spans is a little nasty. It works for all the test cases I threw at it, but I wouldn't be surprised if an edge case would break it. Should have a thorough review. - This PR worsens the state of #56885, since those certain fatal lexing errors are now emitted before docs get generated at all.
|
☀️ Test successful - checks-travis, status-appveyor |
Fixes #53919.
This PR moves the reporting of code block lexing errors from rendering time to an early pass, so we can use the compiler's error reporting mechanisms. This dramatically improves the diagnostics in this situation: we now de-emphasize the lexing errors as a note under a warning that has a span and suggestion instead of just emitting errors at the top level.
Additionally, this PR generalizes the markdown -> source span calculation function, which should allow other rustdoc warnings to use better spans in the future.
Last, the PR makes sure that the code block is always emitted in the docs, even if it fails to highlight correctly.
Of note:
pulldown_cmarkdoesn't make it easy to find the spans of the code blocks, so the code that calculates the spans is a little nasty. It works for all the test cases I threw at it, but I wouldn't be surprised if an edge case would break it. Should have a thorough review.