rustdoc: DOS line endings cause extra newlines to show up in source view

[tesuji]

I tried to document and publish my Windows crate to github pages.
Things went pretty smooth until I opened the source code view.
As you can see below, a newline is appended to each line doc comment,
also no line number for each line:

Here the render page on gh-pages: https://lzutao.github.io/junction/src/junction/lib.rs.html#1-111.
Beware that it has no version control for the generated doc. But the link here may work
for a limited time before github purge the cache: https://github.com/lzutao/junction/tree/6ab9e37b809afdb49fce1accf943250af6aa5dbf. Also here the the CI script I used:
https://github.com/lzutao/junction/blob/724dd2e564c76e4f92580629c2f4a1340875e890/.github/workflows/ci.yml#L62-L95.

The same code published on https://docs.rs is normal: https://docs.rs/junction/0.2.0/src/junction/lib.rs.html#1-111

Meta

rustc --version --verbose: x86_64-pc-windows-msvc rustc 1.48.0-nightly (c59199e 2020-09-04)

@rustbot modify labels: T-rustdoc

[Nemo157]

Could be related to \r\n normalization depending on the git checkout settings of the runner.

[tesuji]

I set git config --global core.autocrlf false before git add, maybe this is my misconfigure somehow.

[AndyGauge]

Try adding git config --global core.eol lf and possibly git add --renormalize . to make those files end with the right character.

[tesuji]

Thanks Andy, I tried tesuji/junction@c8107d0 as you told but still the issue remains.

[Nemo157]

Oh, I meant for the source files, you might need to reconfigure git before checking out the source (and that's likely just a workaround, this seems likely to be a bug in rustdoc's handling of \r\n in source files).

[tesuji]

Thank you, problem is solved in tesuji/junction@005a79c .
I am pretty sure this issue is a duplicate of another, but I cannot find it.

You could close this issue if you see fit.

[Nemo157]

(keywords cause I keep failing to search for this: dos line endings)

[jyn514]

This is a recent regression, it was introduced sometime between rustdoc 1.47.0 (18bf6b4f0 2020-10-07) and rustdoc 1.48.0-beta.1 (bcad406af 2020-10-07). I suspect #75775 is related, cc @matklad.

[jyn514]

The issue is that \r is being marked as a doc-comment:

<span class="doccomment">/// Additional docs
</span>
<!-- this next line ends with \r, not \n -->
<span class="doccomment">///
</span>
<span class="doccomment">/// [LLVM commit D20468][https://reviews.llvm.org/D20468]
</span>

[jyn514]

I guess this means \r is being marked as a LineComment? I don't see why it would.

rust/src/librustdoc/html/highlight.rs

Lines 169 to 171 in 7504256

	TokenKind::LineComment { doc_style } | TokenKind::BlockComment { doc_style, .. } => {
	if doc_style.is_some() {
	Class::DocComment

[jyn514]

This is only happening for /// lines, normal source lines are fine. My test was wrong.

[jyn514]

The issue is that \r is being marked as a doc-comment:

This is not actually correct - it was a blank line in the original source code. The issue is that rustdoc used to strip \r in newlines, but no longer does. Here's 1.47:

<span class="doccomment">/// Additional docs</span>
<span class="doccomment">///</span>
<span class="doccomment">/// [LLVM commit D20468][https://reviews.llvm.org/D20468]</span>

and 1.48:

</pre><div class="example-wrap"><pre class="rust ">
<span class="doccomment">/// Additional docs
</span>
<span class="doccomment">///
</span>
<span class="doccomment">/// [LLVM commit D20468][https://reviews.llvm.org/D20468]
</span>