Implement most of MCP510

[lqd]

This implements most of what remains to be done for MCP510:

• turns -C link-self-contained into a +/- list of components, like -C link-self-contained=+linker,+crto,+libc,+unwind,+sanitizers,+mingw. The scaffolding is present for all these expected components to be implemented and stabilized in the future on their own time. This PR only handles the -Zgcc-ld=lld subset of these link-self-contained components as -Clink-self-contained=+linker
• handles -C link-self-contained=y|n as-is today, for compatibility with rustc_codegen_ssa::back::link::self_contained's explicit opt-in and opt-out.
• therefore supports our plan to opt out of rust-lld (when it's enabled by default) even for current -Clink-self-contained users, with e.g. -Clink-self-contained -Clink-self-contained=-linker
• turns add_gcc_ld_path into its expected final form, by using the -C link-self-contained=+linker CLI flag, and whether the LinkerFlavor has the expected Cc::Yes and Lld::Yes shape (this is not yet the case in practice for any CLI linker flavor)
• makes the new clean linker flavors selectable in the CLI in addition to the legacy ones, in order to opt-in to using cc and lld to emulate -Zgcc-ld=lld
• ensure the new -C link-self-contained components, and -C linker-flavors are unstable, and require -Z unstable-options to be used

The up-to-date set of flags for the future stable CLI version of -Zgcc-ld=lld is currently: -Clink-self-contained=+linker -Clinker-flavor=gnu-lld-cc -Zunstable-options.

It's possible we'll also need to do something for distros that don't ship rust-lld, but maybe there are already no tool search paths to be added to cc in this situation anyways.

r? @petrochenkov

[petrochenkov]

Linking to the previous PRs: #96827 #96884

[petrochenkov]

do you want the CLI to be able to select actual LinkerFlavors (directly, without using LinkerFlavorCli)

I actually planned to implement it next weekend.
So, yes, but not necessarily in this PR.

Like this

pub enum LinkerFlavorCli {
    // New
    Gnu(Cc, Lld),
    Darwin(Cc, Lld),
    WasmLld(Cc),
    Unix(Cc),
    Msvc(Lld),
    EmCc,
    Bpf,
    Ptx,
    // Old
    Gcc,
    Ld,
    Lld(LldFlavor),
    Msvc,
    Em,
    BpfLinker,
    PtxLinker,
}

Then some of the old ones can be removed because they are tier 3, and the rest of them will be supported forever (i.e. this is not even a migration).

Then I wanted to add a couple of new "generic" flavors like *-cc or *-lld-* where the unspecified parts would be inferred from the target spec (the actual user-visible names won't contain *s of course).

[rustbot]

These commits modify compiler targets.
(See the Target Tier Policy.)

[petrochenkov]

This basically looks nearly ready for merging to me.
@rustbot author

[lqd]

I can definitely add the new flavors to LinkerFlavorCli in this PR if you want, and a few tests, to make it a bit closer to usable for real.

I may miss some subtleties around the conversions, although now everything should be automatic, so I understand if you'd prefer doing it yourself.

[petrochenkov]

Full implementation of -C link-self-contained will also require modifying bootstrap to put all the things into corresponding subdirectories (and retrieving from there in rustc).

[petrochenkov]

@lqd
Yes, if you just add the new variants to enum LinkerFlavorCli it will do most of the work.
The only thing I expect breaking is check_compatibility, it may require writing out the match explicitly, because the roundtrip check will no longer work.

[lqd]

Ok great, I'll do that and report back.

[lqd]

I've tried adding the new flavors from the end of your old comment, and today's comment:

Like this

pub enum LinkerFlavorCli {
    // New
    Gnu(Cc, Lld),
    Darwin(Cc, Lld),
    WasmLld(Cc),
    Unix(Cc),
    Msvc(Lld),
    EmCc,
    Bpf,
    Ptx,
    // Old
    Gcc,
    Ld,
    Lld(LldFlavor),
    Msvc,
    Em,
    BpfLinker,
    PtxLinker,
}

A wrinkle was that we have both old and new values for Msvc so I merged the two and consider Msvc(Lld::No) as legacy.
That seemed safe to do, especially when keeping the legacy conversion code mostly as-is; but if you want we could have two different variants.

@rustbot ready

[lqd]

rebased to fix conflicts

[petrochenkov]

It's possible we'll also need to do something for distros that don't ship rust-lld, but maybe there are already no tool search paths to be added to cc in this situation anyways.

I expected this to not require special treatment, even if we add the directory in which rust-lld is usually shipped to search paths, it will be empty and the search will continue to find lld elsewhere on the system.

[lqd]

Agreed, I had the same opinion. We could also check that the build config has rust.lld enabled for -Clinked-self-contained=+linker to work, and otherwise emit an error, like we do for similar cases.

But indeed it might just be a case of verifying that the fallback to another LLD on the $PATH works as we expect -- e.g. by adding a test in the future.

[petrochenkov]

In the end we'll probably have to drop bitflags again to accommodate for target defaults and heuristics.

Like

component1: Option<bool>,
...
componentN: Option<bool>,

Some(true) means explicitly enabled, Some(false) means explicitly disabled, and None means use default heuristics.

[petrochenkov]

Or maybe LinkSelfContainedDefault will use options, and LinkSelfContained keeps using bitflags.
Let's keep things as is for now.

[lqd]

With a prayer to the MIPS gods of Segfaults:

@bors r=petrochenkov

[bors]

📌 Commit 38f5a99 has been approved by petrochenkov

It is now in the queue for this repository.

[bors]

⌛ Testing commit 38f5a99 with merge 8e2d5e3...

[bors]

☀️ Test successful - checks-actions
Approved by: petrochenkov
Pushing 8e2d5e3 to master...

[rust-timer]

Finished benchmarking commit (8e2d5e3): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.4%]	3
Improvements ✅ (secondary)	-1.1%	[-1.2%, -1.0%]	6
All ❌✅ (primary)	-0.4%	[-0.4%, -0.4%]	3

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.4%	[0.5%, 2.7%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.1%	[-2.5%, -1.7%]	2
Improvements ✅ (secondary)	-1.6%	[-1.6%, -1.6%]	1
All ❌✅ (primary)	0.2%	[-2.5%, 2.7%]	6

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	5.4%	[3.7%, 7.2%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 662.486s -> 661.616s (-0.13%)

[lqd]

Note for weekly triage: this doesn't change the compiler in any significant way, and requires opt-in flags, so these diesel and wg-grammar perf wins are noise.