rustup component rust-src should be used to load cross-crate sources.

[eddyb]

If I take this arbitrary example (that supports cross-crate spans, via tcx.def_span(...)):

struct Foo;
impl Extend<()> for Foo {
    fn extend(&mut self, _: impl IntoIterator<Item = ()>) {}
}
fn main() {}

and compiling it with a local rustc build, I get this error (note the libcore snippet):
(NB: if that snippet disappears, find other tcx.def_span(...)-using diagnostics and replace the test)

error[E0643]: method `extend` has incompatible signature for trait
   --> xcrate-span.rs:3:29
    |
3   |     fn extend(&mut self, _: impl IntoIterator<Item = ()>) {}
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected generic parameter, found `impl Trait`
    |
   ::: /home/eddy/Projects/rust-2/src/libcore/iter/traits.rs:355:15
    |
355 |     fn extend<T: IntoIterator<Item=A>>(&mut self, iter: T);
    |               - declaration in trait here

but if try with rustup-provided rustc, I only get this shorter error:

error[E0643]: method `extend` has incompatible signature for trait
   --> xcrate-spans.rs:3:29
    |
3   |     fn extend(&mut self, _: impl IntoIterator<Item = ()>) {}
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected generic parameter, found `impl Trait`

But $(rustc --print=sysroot)/lib/rustlib/src/rust/src/libcore/iter/traits.rs does exist, because I have the rust-src component enabled (it's enabled by default, which makes it likely to exist for most users), so it should be possible in theory, to teach rustc to look up certain paths relative to lib/rustlib/src/rust inside the sysroot, if it exists.

Running this:

strings $(rustc --print=sysroot)/lib/rustlib/*/lib/libcore-*.rlib | rg 'iter/traits\.rs'

shows that /checkout/src/libcore/iter/traits.rs and libcore/iter/traits.rs both exist in the rlib, and I assume the former is the one that it tries to load - we can even test this:

# Let's live a little...
sudo ln -s $(rustc --print=sysroot)/lib/rustlib/src/rust /checkout

Trying the test again, we now get:

error[E0643]: method `extend` has incompatible signature for trait
   --> xcrate-spans.rs:3:29
    |
3   |     fn extend(&mut self, _: impl IntoIterator<Item = ()>) {}
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected generic parameter, found `impl Trait`
    |
   ::: /checkout/src/libcore/iter/traits.rs:355:15
    |
355 |     fn extend<T: IntoIterator<Item=A>>(&mut self, iter: T);
    |               - declaration in trait here

So it's definitely compatible, the hash check passes and whatnot, we just need to rename /checkout to something artificial like $rust, I'm guessing.

# ... but also clean up afterwards.
sudo rm /checkout

cc @alexcrichton @rust-lang/dev-tools @rust-lang/compiler

[alexcrichton]

This sounds like a great idea to me! I think we could definitely bake some logic into rustc such that it has a default src directory for the standard library where, if present, it will connect the spans together. We'd then just have rustup naturally fill the location.

[estebank]

We would have to be careful though, as some diagnostics (specially suggestions) use the definition span being available as a proxy for the end user being able to edit those sources. Because of this, if we implement this improvement without accounting for it we would start suggesting people to modify std, core or even third party crates' code, which we should never do.

[eddyb]

@estebank Right, we'd have to distinguish those more carefully. One thing I should note is that we should be making edit suggestions cross-crate within a workspace, at least IMO.

[estebank]

As soon as we involve other crates, I would reduce the weight of the text from a suggestion to a merely informative explanation of the failure, like we see in your example. We need to add a way to check for this in the diagnostics api. I'm not opposed at all to the idea and think it'd be useful.

[eddyb]

@alexcrichton How does #53829 impact this? Perhaps, for paths starting with /rustc/<hash> (if the hash matches the one the compiler "remembers" as its own?) look in the sysroot for sources?

We'd have to make sure this works for both rustup and distros.

[alexcrichton]

@eddyb I don't think it does? That just affects debuginfo

[eddyb]

Right, but we can use it for recording the paths of files in dependencies as well, right?

[alexcrichton]

Perhaps!

[eddyb]

cc @michaelwoerister

[eddyb]

@alexcrichton I just checked with the latest nightly and it does have an effect:

error[E0643]: method `extend` has incompatible signature for trait
   --> xcrate-spans.rs:3:29
    |
3   |     fn extend(&mut self, _: impl IntoIterator<Item = ()>) {}
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected generic parameter, found `impl Trait`
    |
   ::: /rustc/1f57e4841157d5cbd4c4e22018f93bd1801c98c2/src/libcore/iter/traits.rs:355:15
    |
355 |     fn extend<T: IntoIterator<Item=A>>(&mut self, iter: T);
    |               - declaration in trait here

I had to use (I got that hash with a hex editor from libcore-*.rlib):

sudo mkdir /rustc
sudo ln -s $(rustc --print=sysroot)/lib/rustlib/src/rust /rustc/1f57e4841157d5cbd4c4e22018f93bd1801c98c2

Maybe we can change the lib/rustlib/src/rust path inside the sysroot to either lib/rustlib/src/rustc/<hash> or src/rustc/<hash> (i.e. don't nest it in lib/rustlib)?
Or leave it unchanged and only detect /rustc/<hash>?

Either way, we seem to be set up to handle this gracefully!

[alexcrichton]

I'd be fine with w/e change we need here, should be easy enough to tweak!

[eddyb]

I think we can hardcode the same hash via an env var (like we do for other things, such as compiler version - search for CFG_) in rustc so it can check here that the path (name) starts with /rustc/<harcoded hash passed via CFG_...>/, and replace it:

rust/src/librustc_metadata/decoder.rs

Line 1243 in abe19a7

let local_version = local_source_map.new_imported_source_file(name,

[infinity0]

Would be nice if #56860 was done at the same time as this work.

[saleemjaffer]

@eddyb I would like to take a shot at this. I'm a newbie to rust.

[eddyb]

Oops, I missed this notification, sorry @saleemjaffer!
Ping me on Discord, I suppose, if you're still interested.

[RalfJung]

But $(rustc --print=sysroot)/lib/rustlib/src/rust/src/libcore/iter/traits.rs does exist, because I have the rust-src component enabled (it's enabled by default, which makes it likely to exist for most users)

I don't think that's true, at least not any more.

We would have to be careful though, as some diagnostics (specially suggestions) use the definition span being available as a proxy for the end user being able to edit those sources. Because of this, if we implement this improvement without accounting for it we would start suggesting people to modify std, core or even third party crates' code, which we should never do.

AFAIK we already print span contents for third party crates other than those in the sysroot though, don't we?

[eddyb]

cc @Xanewok Something I noticed lately is that RLS appears to be able to find libstd from rust-src already, I wonder if we could reuse that logic.