Remap compiler vs non-compiler sources differently (bootstrap side) #141751
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustbot
added
the
T-bootstrap
jieyouxu
added
the
A-path-remapping
|
We'll also need in the compiler a different compile-time environment variable, like is currently done for Be aware that both environment variables need to be set when compiling the compiler. |
jieyouxu
changed the title
|
rustbot has assigned @albertlarsan68. Use |
rustbot
added
the
S-waiting-on-review
This comment was marked as resolved.
This comment was marked as resolved.
rustbot
added
S-waiting-on-author
This comment was marked as resolved.
This comment was marked as resolved.
rustbot
added
S-waiting-on-review
- Map compiler sources (corresponding to `rustc-dev` dist component)
with `/rustc-dev/{hash}`.
- Map non-compiler sources (corresponding to `rust-src` dist component
or other non-compiler sources) with `/rustc/{hash}`.
This allows the compiler to have the possibility of opportunistically
reverse the mapping. This is because
- `rust-src` unpacks sources to a path like
`$sysroot/lib/rustlib/src/rust`, whereas
- `rustc-dev` unpacks sources to a path like
`$sysroot/lib/rustlib/rustc-src/rust`
(Notice the `src` vs `rustc-src` difference.)
|
@rustbot ready |
rustbot
added
S-waiting-on-review
|
The changes looks good to me. Thanks you. |
|
r? bootstrap |
|
Thanks for introducing the enum and the comments, it's now much clearer what's going on. @bors r+ |
bors
removed
the
S-waiting-on-review
bors
added
the
S-waiting-on-bors
bors
added a commit
that referenced
this pull request
Jun 9, 2025
Rollup of 6 pull requests Successful merges: - #141751 (Remap compiler vs non-compiler sources differently (bootstrap side)) - #142160 (Only allow `bootstrap` cfg in rustc & related) - #142191 (early return in trait detection for non-trait item) - #142211 (Do not checkout GCC submodule for the tidy job) - #142218 (CI: rfl: move job forward to Linux v6.16-rc1) - #142224 (Avoid a gratuitous 10s wait in a stress test) r? `@ghost` `@rustbot` modify labels: rollup
rust-timer
added a commit
that referenced
this pull request
Jun 9, 2025
Rollup merge of #141751 - jieyouxu:remap, r=Kobzol Remap compiler vs non-compiler sources differently (bootstrap side) See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083). The path remapping and unremapping for compiler sources (distributed via `rustc-dev` dist component) is broken because bootstrap currently remaps all sources unconditionally (if remapping is enabled) to the `/rustc/{hash}` form. However, the `rustc-dev` dist component (compiler sources) and `rust-src` dist component (library sources) unpacks differently: - `rust-src` unpacks sources to a path like `$sysroot/lib/rustlib/src/rust`, whereas - `rustc-dev` unpacks sources to a path like `$sysroot/lib/rustlib/rustc-src/rust`[^note], meaning that the compiler need to unremap them differently. But the same remapping means that the compiler has no way to distinguish between compiler and non-compiler (esp. standard library) sources. To remedy this, this PR adopts the approach of: - remapping compiler sources (corresponding to `rustc-dev` dist component) with `/rustc-dev/{hash}` (this is `RemapScheme::Compiler`), and - remapping non-compiler sources (corresponding to `rust-src` dist component or other non-compiler sources) with `/rustc/{hash}` (this is `RemapScheme::NonCompiler`). A different remapping allows the compiler to reverse the remapping differently. This PR implements the bootstrap side. A follow-up compiler-side change is needed to implement the unremapping change to address the reported issue completely. This PR introduces another env var `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` that is made available to the compiler when building compiler sources to know what the remap scheme for `rustc-dev` (`RemapScheme::Compiler`) is. Compiler sources are built with the compiler remapping scheme. As far as I know, this change should not introduce new regressions, because the compiler source unremapping (through `rustc-dev`) is already broken. [^note]: (Notice the `src` vs `rustc-src` difference.)
fmease
added a commit
to fmease/rust
that referenced
this pull request
Jun 14, 2025
Try unremapping compiler sources See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083). This PR is a follow-up to rust-lang#141751 regarding the compiler side. Specifically we now take into account the `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` env from rust-lang#141751 when trying to unremap sources from `$sysroot/lib/rustlib/rustc-src/rust` (the `rustc-dev` component install directory). Best reviewed commit by commit. cc `@samueltardieu` r? `@jieyouxu`
matthiaskrgr
added a commit
to matthiaskrgr/rust
that referenced
this pull request
Jun 15, 2025
Try unremapping compiler sources See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083). This PR is a follow-up to rust-lang#141751 regarding the compiler side. Specifically we now take into account the `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` env from rust-lang#141751 when trying to unremap sources from `$sysroot/lib/rustlib/rustc-src/rust` (the `rustc-dev` component install directory). Best reviewed commit by commit. cc ``@samueltardieu`` r? ``@jieyouxu``
rust-bors bot
added a commit
that referenced
this pull request
Jun 15, 2025
Try unremapping compiler sources See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083). This PR is a follow-up to #141751 regarding the compiler side. Specifically we now take into account the `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` env from #141751 when trying to unremap sources from `$sysroot/lib/rustlib/rustc-src/rust` (the `rustc-dev` component install directory). Best reviewed commit by commit. cc `@samueltardieu` r? `@jieyouxu` try-job: x86_64-gnu-llvm-20-3
workingjubilee
added a commit
to workingjubilee/rustc
that referenced
this pull request
Jun 17, 2025
Try unremapping compiler sources See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083). This PR is a follow-up to rust-lang#141751 regarding the compiler side. Specifically we now take into account the `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` env from rust-lang#141751 when trying to unremap sources from `$sysroot/lib/rustlib/rustc-src/rust` (the `rustc-dev` component install directory). Best reviewed commit by commit. cc `@samueltardieu` r? `@jieyouxu`
Urgau
added a commit
to Urgau/rust
that referenced
this pull request
Jun 18, 2025
Try unremapping compiler sources See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083). This PR is a follow-up to rust-lang#141751 regarding the compiler side. Specifically we now take into account the `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` env from rust-lang#141751 when trying to unremap sources from `$sysroot/lib/rustlib/rustc-src/rust` (the `rustc-dev` component install directory). Best reviewed commit by commit. cc ``@samueltardieu`` r? ``@jieyouxu``
github-actions bot
pushed a commit
to rust-lang/rustc-dev-guide
that referenced
this pull request
Jun 19, 2025
Try unremapping compiler sources See [#t-compiler/help > Span pointing to wrong file location (`rustc-dev` component)](https://rust-lang.zulipchat.com/#narrow/channel/182449-t-compiler.2Fhelp/topic/Span.20pointing.20to.20wrong.20file.20location.20.28.60rustc-dev.60.20component.29/with/521087083). This PR is a follow-up to rust-lang/rust#141751 regarding the compiler side. Specifically we now take into account the `CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIR` env from rust-lang/rust#141751 when trying to unremap sources from `$sysroot/lib/rustlib/rustc-src/rust` (the `rustc-dev` component install directory). Best reviewed commit by commit. cc ``@samueltardieu`` r? ``@jieyouxu``
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See #t-compiler/help > Span pointing to wrong file location (`rustc-dev` component).
The path remapping and unremapping for compiler sources (distributed via
rustc-devdist component) is broken because bootstrap currently remaps all sources unconditionally (if remapping is enabled) to the/rustc/{hash}form. However, therustc-devdist component (compiler sources) andrust-srcdist component (library sources) unpacks differently:rust-srcunpacks sources to a path like$sysroot/lib/rustlib/src/rust, whereasrustc-devunpacks sources to a path like$sysroot/lib/rustlib/rustc-src/rust1,meaning that the compiler need to unremap them differently. But the same remapping means that the compiler has no way to distinguish between compiler and non-compiler (esp. standard library) sources. To remedy this, this PR adopts the approach of:
rustc-devdist component) with/rustc-dev/{hash}(this isRemapScheme::Compiler), andrust-srcdist component or other non-compiler sources) with/rustc/{hash}(this isRemapScheme::NonCompiler).A different remapping allows the compiler to reverse the remapping differently.
This PR implements the bootstrap side. A follow-up compiler-side change is needed to implement the unremapping change to address the reported issue completely.
This PR introduces another env var
CFG_VIRTUAL_RUSTC_DEV_SOURCE_BASE_DIRthat is made available to the compiler when building compiler sources to know what the remap scheme forrustc-dev(RemapScheme::Compiler) is. Compiler sources are built with the compiler remapping scheme.As far as I know, this change should not introduce new regressions, because the compiler source unremapping (through
rustc-dev) is already broken.Footnotes
(Notice the
srcvsrustc-srcdifference.) ↩