documentation building is sensitive to the order docs are generated

[ehuss]

If documentation for packages is built independently, then any cross-links between packages may not work. It depends on the order of commands, where it is necessary to build from the leaves up.

This is because rustdoc looks in the local directory to decide if it should create a relative link or not. If a dependency hasn't been built yet, then there is nothing for it to find, so it doesn't generate a link.

Usually this is fine for a normal cargo doc command, since it builds from the leaves on up (caveat below). However, if running cargo rustdoc or cargo doc -p foo --no-deps, then the user must know about this subtle interaction to build things in the correct order.

This came up with rust's build scripts here which gets it slightly wrong.

I'm not sure if there is anything that should be done here. One option is to have cargo doc -p foo -p bar be smart and determine the correct order (or create the directories ahead of time). This wouldn't help for rustbuild which uses cargo rustdoc which can only do one package at a time (although it could be changed to cargo doc --no-deps …).

Another option is to just document rustdoc's linking behavior.

Another possible problem is that the dependencies between rustdoc invocations is not explicitly tracked within cargo. This means that if crate A depends on crate B, A and B will be documented concurrently. If A happens to start before B, then A's links to B will be broken. This can be illustrated in the doc builder for rustc here. Removing that create_dir_all causes many links to break. I'm having a hard time creating a small reproduction, though (and I may not fully understand this scenario, since rustc's crate graph is very complex).

cc rust-lang/rust#74352

[kpreid]

Another possible problem is that the dependencies between rustdoc invocations is not explicitly tracked within cargo. This means that if crate A depends on crate B, A and B will be documented concurrently. If A happens to start before B, then A's links to B will be broken. … I'm having a hard time creating a small reproduction, though …

I noticed my cross-crate doc links breaking when I used cargo doc --no-deps, and composed a small repro before finding this issue. Here's a script to generate and test for the bug:

#!/bin/sh

# Run this script in an empty directory that you wish turned into a workspace.

set -ex

cat <<'EOF' > Cargo.toml
[workspace]
members = ["pkg_*"]
EOF

cargo new --lib pkg_foo
cargo new --lib pkg_bar

(cd pkg_bar && cargo add --path ../pkg_foo)

cat <<'EOF' >| pkg_foo/src/lib.rs
#![no_std]
//! This is library `pkg_foo`.
EOF

cat <<'EOF' >| pkg_bar/src/lib.rs
#![no_std]
//! This is library `pkg_bar`, which depends on [`pkg_foo`].
EOF

set +x
echo Testing...

while true; do
  cargo -q clean
  cargo -q doc --no-deps
  grep --count '../pkg_foo/index.html' target/doc/pkg_bar/index.html | head -c 1
done

The output is a sequence of 1s when the proper link is found, and 0 when the link is not found. There are very few 0s in this repro, but in other (more complex) circumstances I would get that effect reliably. Introducing CPU scheduling noise such as unrelated cargo builds seems to help the repro.

[mTsBucy1]

I am wondering why cargo / rustdoc does not generate warnings when it encounters a links that it cannot resolve due to the folder not existing?
Additionally, does no-deps actually also disable intra-workspace dependency tracking, or does it not make a difference for this bug?