Rustdoc doesn't show src button for many functions in the rustc docs

[bjorn3]

https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/map/struct.Map.html

[bjorn3]

@rustbot modify labels:+C-bug +T-rustdoc

[ehuss]

I have narrowed this down to #66364. There some interaction between remap-path-prefix and macros in a dependency.

Repro:

cargo new --lib foo
cd foo
cargo new --lib dep
rm dep/src/lib.rs
cat << EOF > dep/src/lib.rs
#[macro_export]
macro_rules! make_item {
    () => {
        pub fn f() {}
    }
}
EOF

echo 'dep = {path="dep"}' >> Cargo.toml
echo 'dep::make_item!{}' >> src/lib.rs
RUSTFLAGS="--remap-path-prefix=`pwd`=/" cargo doc -v  --no-deps

This emits a warning:

warning: source code was requested to be rendered, but processing `/dep/src/lib.rs` had an error: "/dep/src/lib.rs": No such file or directory (os error 2)

and the [src] links don't exist.

If you want to repro with the rustc sources, set remap-debuginfo = true and compiler-docs = true in config.toml and build the rustc docs.

@Centril can you maybe take a look at why this is now broken?

[Centril]

For now, I'll cc @GuillaumeGomez @eddyb @petrochenkov who probably know more about this than I.

[Centril]

Also relevant:

rust/src/librustdoc/html/sources.rs

Lines 35 to 55 in 2113659

	// If we're including source files, and we haven't seen this file yet,
	// then we need to render it out to the filesystem.
	if self.scx.include_sources
	// skip all synthetic "files"
	&& item.source.filename.is_real()
	// skip non-local items
	&& item.def_id.is_local()
	{
	// If it turns out that we couldn't read this file, then we probably
	// can't read any of the files (generating html output from json or
	// something like that), so just don't include sources for the
	// entire crate. The other option is maintaining this mapping on a
	// per-file basis, but that's probably not worth it...
	self.scx.include_sources = match self.emit_source(&item.source.filename) {
	Ok(()) => true,
	Err(e) => {
	println!(
	"warning: source code was requested to be rendered, \
	but processing `{}` had an error: {}",
	item.source.filename, e
	);

[eddyb]

Hang on, but there should be no macros involved in the impl @bjorn3 posted, how is #66364 relevant?

[bjorn3]

That code sets self.scx.include_sources to false when any file could not be read.

[eddyb]

Oh, that's just bad UX and should be fixed independently on not being able to load some file.

[eddyb]

@Mark-Simulacrum Is there any way to use the try build on #70642 to check if it fixes this issue?
i.e. does it include any docs?

[Mark-Simulacrum]

I think it should, but not sure. We have a try build going and once that's done IIRC a command like rustup-toolchain-install-master 40f2b445cd1c6c4e72d663d23c6d1bef2c674ad4 -c rust-docs should work, I believe.

[eddyb]

@Mark-Simulacrum Uh oh I can't get anything like https://doc.rust-lang.org/nightly/nightly-rustc even for regular nightly in rustup, and none of alloc/core/proc_macro/std/test are affected.

Also, since this seems to affect the whole crate, it's easier to spot at that level:

• Affected crates that use the newtype_index! macro to define types:
  • rustc_ast
  • rustc_hir
  • rustc_middle
  • rustc_mir
  • rustc_mir_build
  • rustc_query_system
  • rustc_span
  • rustc_target
• Other affected crates, that use declare_lint_pass! to define types:
  • rustc_lint
• Other affected crates, that use declare_box_region_type! to define types:
  • rustc_interface
• Other affected crates, that use impl_stable_hash_via_hash! to define impls:
  • rustc_errors
  • rustc_session
• Other affected crates, that use implement_ty_decoder! to define impls:
  • rustc_metadata
• Other affected crates, that use thread_local! to define statics:
  • rustc_data_structures
  • rustdoc

And that's all the affected crates! Each uses macros from other crates, which define something.

---

thread_local! stands out as being from the standard library. Sure enough:

thread_local!(pub static FOO: bool = false);

$ cargo doc
 Documenting test-70025 v0.1.0 (/home/eddy/Projects/test-70025)
warning: source code was requested to be rendered, but processing `/rustc/76b11980ad416c3ad6143504c2277757ecacf9b5/src/libstd/thread/local.rs` had an error: "/rustc/76b11980ad416c3ad6143504c2277757ecacf9b5/src/libstd/thread/local.rs": No such file or directory (os error 2)
         skipping rendering of source code
    Finished dev [unoptimized + debuginfo] target(s) in 3.53s

But with #70642, it works:

$ rustup-toolchain-install-master 40f2b445cd1c6c4e72d663d23c6d1bef2c674ad4 -c rust-src
$ cargo +40f2b445cd1c6c4e72d663d23c6d1bef2c674ad4 doc
 Documenting test-70025 v0.1.0 (/home/eddy/Projects/test-70025)
    Finished dev [unoptimized + debuginfo] target(s) in 2.21s
$ find target/doc/src/ | rg html
target/doc/src/test_70025/lib.rs.html
target/doc/src/test_70025/home/eddy/.rustup/toolchains/40f2b445cd1c6c4e72d663d23c6d1bef2c674ad4/lib/rustlib/src/rust/src/libstd/thread/local.rs.html

However, I have no way to check the CI environment itself, so we'll have to wait for #70642 to land.

[eddyb]

Something rustdoc could be doing, though, is checking which CrateNum a file is imported from, and try to use that crate's doc URLs, or at the very least use a separate doc/src dir, e.g. instead of:

• target/doc/src/test_70025/home/eddy/.rustup/toolchains/40f2b445cd1c6c4e72d663d23c6d1bef2c674ad4/lib/rustlib/src/rust/src/libstd/thread/local.rs.html

it should probably have created:

• target/doc/src/std/thread/local.rs.html

[eddyb]

#70642 landed, let's see what happens in 14-15h from now.

[eddyb]

And the source is back! So I'm closing this issue, and I've opened #70757 for the remaining items.