debuginfo: Generalize C++-like encoding for enums. #98393

michaelwoerister · 2022-06-22T16:21:13Z

The updated encoding should be able to handle niche layouts where more than one variant has fields (as introduced in #94075).

The new encoding is more uniform as there is no structural difference between direct-tag, niche-tag, and no-tag layouts anymore. The only difference between those cases is that the "dataful" variant in a niche-tag enum will have a (start, end) pair denoting the tag range instead of a single value.

The new encoding now also supports 128-bit tags, which occur in at least some standard library types. These tags are represented as u64 pairs so that debuggers (which don't always have support for 128-bit integers) can reliably deal with them. The downside is that this adds quite a bit of complexity to the encoding and especially to the corresponding NatVis.

The new encoding seems to increase the size of (x86_64-pc-windows-msvc) debuginfo by 10-15%. The size of binaries is not affected (release builds were built with -Cdebuginfo=2, numbers are in kilobytes):

EXE	before	after	relative
cargo (debug)	40453	40450	+0%
ripgrep (debug)	10275	10273	+0%
cargo (release)	16186	16185	+0%
ripgrep (release)	4727	4726	+0%

PDB	before	after	relative
cargo (debug)	236524	261412	+11%
ripgrep (debug)	53140	59060	+11%
cargo (release)	148516	169620	+14%
ripgrep (release)	10676	11804	+11%

Given that the new encoding is more general, this is to be expected. Only platforms using C++-like debuginfo are affected -- which currently is only *-pc-windows-msvc.

TODO

Properly update documentation
Add regression tests for new optimized enum layouts as introduced by Use niche-filling optimization even when multiple variants have data. #94075.

r? @wesleywiser

michaelwoerister · 2022-07-01T12:03:40Z

@bors try @rust-timer queue

rust-timer · 2022-07-01T12:03:44Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-07-01T12:03:51Z

⌛ Trying commit 64d9011ab107b94f46d6ce7df0f0c9e4d92ae029 with merge c10a6d5629fa5fe05b77feda456310dc9e3778a5...

bors · 2022-07-01T13:56:21Z

☀️ Try build successful - checks-actions
Build commit: c10a6d5629fa5fe05b77feda456310dc9e3778a5 (c10a6d5629fa5fe05b77feda456310dc9e3778a5)

rust-timer · 2022-07-01T13:56:25Z

Queued c10a6d5629fa5fe05b77feda456310dc9e3778a5 with parent 7e2733b, future comparison URL.

rust-timer · 2022-07-01T15:13:36Z

Finished benchmarking commit (c10a6d5629fa5fe05b77feda456310dc9e3778a5): comparison url.

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

Primary benchmarks: 🎉 relevant improvement found
Secondary benchmarks: 🎉 relevant improvements found

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	N/A	N/A	0
Improvements 🎉 (primary)	-6.2%	-6.2%	1
Improvements 🎉 (secondary)	-3.0%	-4.2%	3
All 😿🎉 (primary)	-6.2%	-6.2%	1

Cycles

Results

Primary benchmarks: 😿 relevant regression found
Secondary benchmarks: mixed results

	mean¹	max	count²
Regressions 😿 (primary)	2.6%	2.6%	1
Regressions 😿 (secondary)	2.3%	2.3%	1
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	-2.9%	-2.9%	1
All 😿🎉 (primary)	2.6%	2.6%	1

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf -perf-regression

the arithmetic mean of the percent change ↩ ↩²
number of relevant changes ↩ ↩²

compiler/rustc_codegen_llvm/src/debuginfo/metadata/enums/cpp_like.rs

vadimcn · 2022-07-07T17:51:32Z

compiler/rustc_codegen_llvm/src/debuginfo/metadata/enums/cpp_like.rs

+///   int_type tag;
+///
+///   enum VariantNames {
+///      <name-of-variant-0> = 0, // The numeric values are variant index,


This comment is kinda misleading, because it implies that in order to enumerate all variants in an enum one could simply take all values of VariantNames and obtain variant filed names by concatenating variant and the variant index. However variant_fallback throws a wrench into this plan.
So how is one supposed to enumerate enum variants? Should I simply look for all top-level members that begin with variant?

The comment is accurate. The VariantNames enum will contain the names of all variants. You can use it to enumerate them.

I meant that it's easy to assume that "variant index" here is the same as X in variantX. At least I did assume that upon first reading.

Yes, that's correct: both the X in the variantX field name and the numeric values of the VariantNames enum correspond to the variant index - i.e. the zero-based index of the variant as listed in the enums source code definition.

vadimcn · 2022-07-07T17:56:30Z

compiler/rustc_codegen_llvm/src/debuginfo/metadata/enums/cpp_like.rs

+/// 128-bit integers, so all values involved get split into two 64-bit fields.
+/// Instead of the `tag` field, we generate two fields `tag128_lo` and `tag128_hi`,
+/// Instead of `DISCR_EXACT`, we generate `DISCR128_EXACT_LO` and `DISCR128_EXACT_HI`,
+/// and so on.


Other questions I have:

Should I expect to deal with more than one variant using tag range? Or is it always just the variant_fallback? Could this change in the future?

Are variantX.NAMEand the name of the type of variantX.value supposed to stay in sync? Should one be preferred to the other when displaying variant name to the user?

Is tag's int_type guaranteed to be unsigned?

How is one supposed to know when to switch to 128-bit tag/constants? By checking whether tag or tag128... is present?

To resolve these ambiguities, I think it would be helpful to include pseudo-code for decoding enums. I know this was geared towards declarative natvis, but not all debuggers use natvis. As a debugger author, I would appreciate that very much!

Should I expect to deal with more than one variant using tag range? Or is it always just the variant_fallback?

At the moment it's only variant_fallback.

Could this change in the future?

I'm not aware of any concrete plans to change that -- but it could change in the future, yes.

Are variantX.NAME and the name of the type of variantX.value supposed to stay in sync? Should one be preferred to the other when displaying variant name to the user?

Yes, they'll stay in sync. You can use whatever is easier to implement.

Is tag's int_type guaranteed to be unsigned?

No, it can be signed. I'm not a quite clear about the situation for niche-tag enums, though. The compiler currently will only describe those as unsigned types in debuginfo -- but I don't know if that's actually correct in all cases.

How is one supposed to know when to switch to 128-bit tag/constants? By checking whether tag or tag128... is present?

Yes, if there is a tag128_* field in the top-level union, it's 128-bits. Otherwise, there'll be a tag field with some other integer type.

To resolve these ambiguities, I think it would be helpful to include pseudo-code for decoding enums. I know this was geared towards declarative natvis, but not all debuggers use natvis. As a debugger author, I would appreciate that very much!

I'll add something.

compiler/rustc_codegen_llvm/src/debuginfo/metadata/enums/mod.rs

src/etc/natvis/intrinsic.natvis

wesleywiser · 2022-07-07T20:08:27Z

src/etc/natvis/intrinsic.natvis

+
+    <Intrinsic Name="in_range128" Expression="(lt128(begin_hi, begin_lo, end_hi, end_lo)) ?
+                                              (lt_or_eq128(begin_hi, begin_lo, tag128_hi, tag128_lo) &amp;&amp; lt_or_eq128(tag128_hi, tag128_lo, end_hi, end_lo)) :
+                                              (lt_or_eq128(begin_hi, begin_lo, tag128_hi, tag128_lo) || lt_or_eq128(tag128_hi, tag128_lo, end_hi, end_lo))">


Is this part right? If begin > end, don't we need to do tag > start || tag < end?

It should be equivalent: (tag >= start) == (start <= tag). I switched from (tag >= start) to (start <= tag) so I don't have to create a gt_or_eq128 function.

michaelwoerister · 2022-07-08T11:22:48Z

So, I'm somewhat conflicted about variant_fallback for niche-layout enums. As @vadimcn's comment make clear, it's something that makes it harder to reason about the encoding. And at the same time, it limits the encoding to there only being a single variant that corresponds to a range of tag values.

The (only) reason it does exist is to make the (already very big) NatVis definition smaller. With variant_fallback we have:

<DisplayString Condition="tag == variant0.DISCR_EXACT" Optional="true">{variant0.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant1.DISCR_EXACT" Optional="true">{variant1.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant2.DISCR_EXACT" Optional="true">{variant2.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant3.DISCR_EXACT" Optional="true">{variant3.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant4.DISCR_EXACT" Optional="true">{variant4.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant5.DISCR_EXACT" Optional="true">{variant5.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant6.DISCR_EXACT" Optional="true">{variant6.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant7.DISCR_EXACT" Optional="true">{variant7.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant8.DISCR_EXACT" Optional="true">{variant8.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant9.DISCR_EXACT" Optional="true">{variant9.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant10.DISCR_EXACT" Optional="true">{variant10.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant11.DISCR_EXACT" Optional="true">{variant11.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant12.DISCR_EXACT" Optional="true">{variant12.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant13.DISCR_EXACT" Optional="true">{variant13.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant14.DISCR_EXACT" Optional="true">{variant14.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant15.DISCR_EXACT" Optional="true">{variant15.NAME,en}</DisplayString>

<DisplayString Condition="in_range(tag, variant_fallback.DISCR_BEGIN, variant_fallback.DISCR_END)" 
               Optional="true">{variant_fallback.NAME,en}</DisplayString>

Without it, we have to check for every variant if it corresponds to a range, so we get:

<DisplayString Condition="tag == variant0.DISCR_EXACT" Optional="true">{variant0.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant1.DISCR_EXACT" Optional="true">{variant1.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant2.DISCR_EXACT" Optional="true">{variant2.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant3.DISCR_EXACT" Optional="true">{variant3.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant4.DISCR_EXACT" Optional="true">{variant4.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant5.DISCR_EXACT" Optional="true">{variant5.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant6.DISCR_EXACT" Optional="true">{variant6.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant7.DISCR_EXACT" Optional="true">{variant7.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant8.DISCR_EXACT" Optional="true">{variant8.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant9.DISCR_EXACT" Optional="true">{variant9.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant10.DISCR_EXACT" Optional="true">{variant10.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant11.DISCR_EXACT" Optional="true">{variant11.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant12.DISCR_EXACT" Optional="true">{variant12.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant13.DISCR_EXACT" Optional="true">{variant13.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant14.DISCR_EXACT" Optional="true">{variant14.NAME,en}</DisplayString>
<DisplayString Condition="tag == variant15.DISCR_EXACT" Optional="true">{variant15.NAME,en}</DisplayString>

<DisplayString Condition="in_range(tag, variant0.DISCR_BEGIN, variant0.DISCR_END)" Optional="true">{variant0.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant1.DISCR_BEGIN, variant1.DISCR_END)" Optional="true">{variant1.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant2.DISCR_BEGIN, variant2.DISCR_END)" Optional="true">{variant2.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant3.DISCR_BEGIN, variant3.DISCR_END)" Optional="true">{variant3.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant4.DISCR_BEGIN, variant4.DISCR_END)" Optional="true">{variant4.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant5.DISCR_BEGIN, variant5.DISCR_END)" Optional="true">{variant5.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant6.DISCR_BEGIN, variant6.DISCR_END)" Optional="true">{variant6.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant7.DISCR_BEGIN, variant7.DISCR_END)" Optional="true">{variant7.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant8.DISCR_BEGIN, variant8.DISCR_END)" Optional="true">{variant8.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant9.DISCR_BEGIN, variant9.DISCR_END)" Optional="true">{variant9.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant10.DISCR_BEGIN, variant10.DISCR_END)" Optional="true">{variant10.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant11.DISCR_BEGIN, variant11.DISCR_END)" Optional="true">{variant11.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant12.DISCR_BEGIN, variant12.DISCR_END)" Optional="true">{variant12.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant13.DISCR_BEGIN, variant13.DISCR_END)" Optional="true">{variant13.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant14.DISCR_BEGIN, variant14.DISCR_END)" Optional="true">{variant14.NAME,en}</DisplayString>
<DisplayString Condition="in_range(tag, variant15.DISCR_BEGIN, variant15.DISCR_END)" Optional="true">{variant15.NAME,en}</DisplayString>

And there are actually another three similar copies of these two blocks to account for 128-bit tags and field expansion. It's quite the monstrosity😅

But I'm not sure that that would actually be a problem. On the one hand, I'm worried about NatVis evaluation engines not being built for visualizers of this size. On the other hand, I haven't seen any actual evidence that either Visual Studio or WinDbg have trouble interpreting these.

So I'm not sure. Shall we make the encoding more regular and consistent by removing variant_fallback? I'm starting to be in favor of that.

michaelwoerister · 2022-07-08T15:25:40Z

I've removed variant_fallback now. The NatVis is more verbose, but the encoding is more regular.

vadimcn · 2022-07-08T19:19:02Z

compiler/rustc_codegen_llvm/src/debuginfo/metadata/enums/cpp_like.rs

+///   int_type tag;
+///
+///   enum VariantNames {
+///      <name-of-variant-0> = 0, // The numeric values are variant index,


I meant that it's easy to assume that "variant index" here is the same as X in variantX. At least I did assume that upon first reading.

compiler/rustc_codegen_llvm/src/debuginfo/metadata/enums/cpp_like.rs

…debuginfo encoding.

wesleywiser · 2022-08-12T16:24:42Z

@bors r+

bors · 2022-08-12T16:24:43Z

📌 Commit 6d030a5 has been approved by wesleywiser

It is now in the queue for this repository.

bors · 2022-08-12T19:13:13Z

⌛ Testing commit 6d030a5 with merge 8f836d90641b30d3d128f5f912a697915bd1178e...

bors · 2022-08-12T19:53:42Z

💔 Test failed - checks-actions

wesleywiser

I didn't spot any other instances of usize issues :)

src/test/debuginfo/msvc-pretty-enums.rs

michaelwoerister · 2022-08-15T10:31:43Z

Tests updated.

@bors r=wesleywiser

bors · 2022-08-15T10:31:44Z

📌 Commit 03b93d0 has been approved by wesleywiser

It is now in the queue for this repository.

bors · 2022-08-15T12:59:56Z

⌛ Testing commit 03b93d0 with merge 4916e2b...

bors · 2022-08-15T15:25:26Z

☀️ Test successful - checks-actions
Approved by: wesleywiser
Pushing 4916e2b to master...

rust-timer · 2022-08-15T16:40:51Z

Finished benchmarking commit (4916e2b): comparison url.

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

Primary benchmarks: ❌ relevant regression found
Secondary benchmarks: mixed results

( )	mean¹	max	count²
Regressions ❌ (primary)	2.3%	2.3%	1
Regressions ❌ (secondary)	4.5%	4.5%	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.1%	-2.1%	1
All ❌✅ (primary)	2.3%	2.3%	1

Cycles

Results

Primary benchmarks: ✅ relevant improvement found
Secondary benchmarks: ❌ relevant regressions found

( )	mean¹	max	count²
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.6%	2.6%	2
Improvements ✅ (primary)	-3.0%	-3.0%	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-3.0%	-3.0%	1

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

the arithmetic mean of the percent change ↩ ↩²
number of relevant changes ↩ ↩²

Pkgsrc changes: * We now manage to build for mipsel-unknown-netbsd, but despite the target spec saying cpu = "mips3", the compiler manages to emit 64-bit instructions which cause "illegal instruction" error. Will need more work. The mipsel-unknown-netbsd entry is commentd out since there is no 1.64.0 bootstrap. * Managed to retain the build of aarch64_be, llvm needed a patch to avoid use of neon instructions in the BE case (llvm doesn't support use of neon in BE mode). Ref. patch to src/llvm-project/llvm/lib/Support/BLAKE3/blake3_impl.h. * The minimum gcc version is now 7.x, and that includes the cross-compiler for the targets. For i386 this also needs to /usr/include/gcc-7 include files in the target root, because immintrin.h from gcc 5 is not compatible with gcc 7.x. This applies for the targets where we build against a root from netbsd-8 (sparc64, powerpc, i386), and files/gcc-wrap gets a hack for this. * Pick up tweak for -latomic inclusion from rust-lang/rust#104220 and rust-lang/rust#104572 * Retain ability to do 32-bit NetBSD, by changing from 64 to 32 bit types in library/std/src/sys/unix/thread_parker/netbsd.rs. * I've struggled a bit to get the "openssl-src" build with -latomic where it's needed. I introduce "NetBSD-generic32" system type and use it for the NetBSD mipsel target. There is another attempt to do the same in the patch to vendor/openssl-sys/build/main.rs. * Bump bootstraps to 1.64.0, checksum updates. Upstream changes: Version 1.65.0 (2022-11-03) ========================== Language -------- - [Error on `as` casts of enums with `#[non_exhaustive]` variants] (rust-lang/rust#92744) - [Stabilize `let else`](rust-lang/rust#93628) - [Stabilize generic associated types (GATs)] (rust-lang/rust#96709) - [Add lints `let_underscore_drop`, `let_underscore_lock`, and `let_underscore_must_use` from Clippy] (rust-lang/rust#97739) - [Stabilize `break`ing from arbitrary labeled blocks ("label-break-value")] (rust-lang/rust#99332) - [Uninitialized integers, floats, and raw pointers are now considered immediate UB](rust-lang/rust#98919). Usage of `MaybeUninit` is the correct way to work with uninitialized memory. - [Stabilize raw-dylib for Windows x86_64, aarch64, and thumbv7a] (rust-lang/rust#99916) - [Do not allow `Drop` impl on foreign ADTs] (rust-lang/rust#99576) Compiler -------- - [Stabilize -Csplit-debuginfo on Linux] (rust-lang/rust#98051) - [Use niche-filling optimization even when multiple variants have data] (rust-lang/rust#94075) - [Associated type projections are now verified to be well-formed prior to resolving the underlying type] (rust-lang/rust#99217) - [Stringify non-shorthand visibility correctly] (rust-lang/rust#100350) - [Normalize struct field types when unsizing] (rust-lang/rust#101831) - [Update to LLVM 15](rust-lang/rust#99464) - [Fix aarch64 call abi to correctly zeroext when needed] (rust-lang/rust#97800) - [debuginfo: Generalize C++-like encoding for enums] (rust-lang/rust#98393) - [Add `special_module_name` lint] (rust-lang/rust#94467) - [Add support for generating unique profraw files by default when using `-C instrument-coverage`] (rust-lang/rust#100384) - [Allow dynamic linking for iOS/tvOS targets] (rust-lang/rust#100636) New targets: - [Add armv4t-none-eabi as a tier 3 target] (rust-lang/rust#100244) - [Add powerpc64-unknown-openbsd and riscv64-unknown-openbsd as tier 3 targets] (rust-lang/rust#101025) - Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Don't generate `PartialEq::ne` in derive(PartialEq)] (rust-lang/rust#98655) - [Windows RNG: Use `BCRYPT_RNG_ALG_HANDLE` by default] (rust-lang/rust#101325) - [Forbid mixing `System` with direct system allocator calls] (rust-lang/rust#101394) - [Document no support for writing to non-blocking stdio/stderr] (rust-lang/rust#101416) - [`std::layout::Layout` size must not overflow `isize::MAX` when rounded up to `align`](rust-lang/rust#95295) This also changes the safety conditions on `Layout::from_size_align_unchecked`. Stabilized APIs --------------- - [`std::backtrace::Backtrace`] (https://doc.rust-lang.org/stable/std/backtrace/struct.Backtrace.html) - [`Bound::as_ref`] (https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.as_ref) - [`std::io::read_to_string`] (https://doc.rust-lang.org/stable/std/io/fn.read_to_string.html) - [`<*const T>::cast_mut`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_mut) - [`<*mut T>::cast_const`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_const) These APIs are now stable in const contexts: - [`<*const T>::offset_from`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from) - [`<*mut T>::offset_from`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from) Cargo ----- - [Apply GitHub fast path even for partial hashes] (rust-lang/cargo#10807) - [Do not add home bin path to PATH if it's already there] (rust-lang/cargo#11023) - [Take priority into account within the pending queue] (rust-lang/cargo#11032). This slightly optimizes job scheduling by Cargo, with typically small improvements on larger crate graph builds. Compatibility Notes ------------------- - [`std::layout::Layout` size must not overflow `isize::MAX` when rounded up to `align`] (rust-lang/rust#95295). This also changes the safety conditions on `Layout::from_size_align_unchecked`. - [`PollFn` now only implements `Unpin` if the closure is `Unpin`] (rust-lang/rust#102737). This is a possible breaking change if users were relying on the blanket unpin implementation. See discussion on the PR for details of why this change was made. - [Drop ExactSizeIterator impl from std::char::EscapeAscii] (rust-lang/rust#99880) This is a backwards-incompatible change to the standard library's surface area, but is unlikely to affect real world usage. - [Do not consider a single repeated lifetime eligible for elision in the return type] (rust-lang/rust#103450) This behavior was unintentionally changed in 1.64.0, and this release reverts that change by making this an error again. - [Reenable disabled early syntax gates as future-incompatibility lints] (rust-lang/rust#99935) - [Update the minimum external LLVM to 13] (rust-lang/rust#100460) - [Don't duplicate file descriptors into stdio fds] (rust-lang/rust#101426) - [Sunset RLS](rust-lang/rust#100863) - [Deny usage of `#![cfg_attr(..., crate_type = ...)]` to set the crate type] (rust-lang/rust#99784) This strengthens the forward compatibility lint deprecated_cfg_attr_crate_type_name to deny. - [`llvm-has-rust-patches` allows setting the build system to treat the LLVM as having Rust-specific patches] (rust-lang/rust#101072) This option may need to be set for distributions that are building Rust with a patched LLVM via `llvm-config`, not the built-in LLVM. Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Add `x.sh` and `x.ps1` shell scripts] (rust-lang/rust#99992) - [compiletest: use target cfg instead of hard-coded tables] (rust-lang/rust#100260) - [Use object instead of LLVM for reading bitcode from rlibs] (rust-lang/rust#98100) - [Enable MIR inlining for optimized compilations] (rust-lang/rust#91743) This provides a 3-10% improvement in compiletimes for real world crates. See [perf results] (https://perf.rust-lang.org/compare.html?start=aedf78e56b2279cc869962feac5153b6ba7001ed&end=0075bb4fad68e64b6d1be06bf2db366c30bc75e1&stat=instructions:u).

Pkgsrc changes: * pkglint cleanups, bump bootstrap kits to 1.65.0. * New target: mipsel-unknown-netbsd, for cpu=mips32 with soft-float. * Managed to retain the build of aarch64_be, llvm needed a patch to avoid use of neon instructions in the BE case (llvm doesn't support use of neon in BE mode). Ref. patch to src/llvm-project/llvm/lib/Support/BLAKE3/blake3_impl.h. Also submitted upstream of LLVM to the BLAKE3 maintainers. * The minimum gcc version is now 7.x, and that includes the cross-compiler for the targets. For i386 this also needs to /usr/include/gcc-7 include files in the target root, because immintrin.h from gcc 5 is not compatible with gcc 7.x. This applies for the targets where we build against a root from netbsd-8 (sparc64, powerpc, i386), and files/gcc-wrap gets a hack for this. * Pick up tweak for -latomic inclusion from rust-lang/rust#104220 and rust-lang/rust#104572 * Retain ability to do 32-bit NetBSD, by changing from 64 to 32 bit types in library/std/src/sys/unix/thread_parker/netbsd.rs. * I've tried to get the "openssl-src" build with -latomic where it's needed. I've introduced the "NetBSD-generic32" system type and use it for the NetBSD mipsel target. There is another attempt to do the same in the patch to vendor/openssl-sys/build/main.rs. Upstream changes: Version 1.66.1 (2023-01-10) =========================== - Added validation of SSH host keys for git URLs in Cargo ([CVE-2022-46176](https://www.cve.org/CVERecord?id=CVE-2022-46176)) Version 1.66.0 (2022-12-15) =========================== Language -------- - [Permit specifying explicit discriminants on all `repr(Int)` enums](rust-lang/rust#95710) ```rust #[repr(u8)] enum Foo { A(u8) = 0, B(i8) = 1, C(bool) = 42, } ``` - [Allow transmutes between the same type differing only in lifetimes](rust-lang/rust#101520) - [Change constant evaluation errors from a deny-by-default lint to a hard error](rust-lang/rust#102091) - [Trigger `must_use` on `impl Trait` for supertraits](rust-lang/rust#102287) This makes `impl ExactSizeIterator` respect the existing `#[must_use]` annotation on `Iterator`. - [Allow `..X` and `..=X` in patterns](rust-lang/rust#102275) - [Uplift `clippy::for_loops_over_fallibles` lint into rustc](rust-lang/rust#99696) - [Stabilize `sym` operands in inline assembly](rust-lang/rust#103168) - [Update to Unicode 15](rust-lang/rust#101912) - [Opaque types no longer imply lifetime bounds](rust-lang/rust#95474) This is a soundness fix which may break code that was erroneously relying on this behavior. Compiler -------- - [Add armv5te-none-eabi and thumbv5te-none-eabi tier 3 targets](rust-lang/rust#101329) - Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. - [Add support for linking against macOS universal libraries](rust-lang/rust#98736) Libraries --------- - [Fix `#[derive(Default)]` on a generic `#[default]` enum adding unnecessary `Default` bounds](rust-lang/rust#101040) - [Update to Unicode 15](rust-lang/rust#101821) Stabilized APIs --------------- - [`proc_macro::Span::source_text`](https://doc.rust-lang.org/stable/proc_macro/struct.Span.html#method.source_text) - [`uX::{checked_add_signed, overflowing_add_signed, saturating_add_signed, wrapping_add_signed}`](https://doc.rust-lang.org/stable/std/primitive.u8.html#method.checked_add_signed) - [`iX::{checked_add_unsigned, overflowing_add_unsigned, saturating_add_unsigned, wrapping_add_unsigned}`](https://doc.rust-lang.org/stable/std/primitive.i8.html#method.checked_add_unsigned) - [`iX::{checked_sub_unsigned, overflowing_sub_unsigned, saturating_sub_unsigned, wrapping_sub_unsigned}`](https://doc.rust-lang.org/stable/std/primitive.i8.html#method.checked_sub_unsigned) - [`BTreeSet::{first, last, pop_first, pop_last}`](https://doc.rust-lang.org/stable/std/collections/struct.BTreeSet.html#method.first) - [`BTreeMap::{first_key_value, last_key_value, first_entry, last_entry, pop_first, pop_last}`](https://doc.rust-lang.org/stable/std/collections/struct.BTreeMap.html#method.first_key_value) - [Add `AsFd` implementations for stdio lock types on WASI.](rust-lang/rust#101768) - [`impl TryFrom<Vec<T>> for Box<[T; N]>`](https://doc.rust-lang.org/stable/std/boxed/struct.Box.html#impl-TryFrom%3CVec%3CT%2C%20Global%3E%3E-for-Box%3C%5BT%3B%20N%5D%2C%20Global%3E) - [`core::hint::black_box`](https://doc.rust-lang.org/stable/std/hint/fn.black_box.html) - [`Duration::try_from_secs_{f32,f64}`](https://doc.rust-lang.org/stable/std/time/struct.Duration.html#method.try_from_secs_f32) - [`Option::unzip`](https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.unzip) - [`std::os::fd`](https://doc.rust-lang.org/stable/std/os/fd/index.html) Rustdoc ------- - [Add Rustdoc warning for invalid HTML tags in the documentation](rust-lang/rust#101720) Cargo ----- - [Added `cargo remove` to remove dependencies from Cargo.toml](https://doc.rust-lang.org/nightly/cargo/commands/cargo-remove.html) - [`cargo publish` now waits for the new version to be downloadable before exiting](rust-lang/cargo#11062) See [detailed release notes](https://github.com/rust-lang/cargo/blob/master/CHANGELOG.md#cargo-166-2022-12-15) for more. Compatibility Notes ------------------- - [Only apply `ProceduralMasquerade` hack to older versions of `rental`](rust-lang/rust#94063) - [Don't export `__heap_base` and `__data_end` on wasm32-wasi.](rust-lang/rust#102385) - [Don't export `__wasm_init_memory` on WebAssembly.](rust-lang/rust#102426) - [Only export `__tls_*` on wasm32-unknown-unknown.](rust-lang/rust#102440) - [Don't link to `libresolv` in libstd on Darwin](rust-lang/rust#102766) - [Update libstd's libc to 0.2.135 (to make `libstd` no longer pull in `libiconv.dylib` on Darwin)](rust-lang/rust#103277) - [Opaque types no longer imply lifetime bounds](rust-lang/rust#95474) This is a soundness fix which may break code that was erroneously relying on this behavior. - [Make `order_dependent_trait_objects` show up in future-breakage reports](rust-lang/rust#102635) - [Change std::process::Command spawning to default to inheriting the parent's signal mask](rust-lang/rust#101077) Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Enable BOLT for LLVM compilation](rust-lang/rust#94381) - [Enable LTO for rustc_driver.so](rust-lang/rust#101403) Version 1.65.0 (2022-11-03) ========================== Language -------- - [Error on `as` casts of enums with `#[non_exhaustive]` variants] (rust-lang/rust#92744) - [Stabilize `let else`](rust-lang/rust#93628) - [Stabilize generic associated types (GATs)] (rust-lang/rust#96709) - [Add lints `let_underscore_drop`, `let_underscore_lock`, and `let_underscore_must_use` from Clippy] (rust-lang/rust#97739) - [Stabilize `break`ing from arbitrary labeled blocks ("label-break-value")] (rust-lang/rust#99332) - [Uninitialized integers, floats, and raw pointers are now considered immediate UB](rust-lang/rust#98919). Usage of `MaybeUninit` is the correct way to work with uninitialized memory. - [Stabilize raw-dylib for Windows x86_64, aarch64, and thumbv7a] (rust-lang/rust#99916) - [Do not allow `Drop` impl on foreign ADTs] (rust-lang/rust#99576) Compiler -------- - [Stabilize -Csplit-debuginfo on Linux] (rust-lang/rust#98051) - [Use niche-filling optimization even when multiple variants have data] (rust-lang/rust#94075) - [Associated type projections are now verified to be well-formed prior to resolving the underlying type] (rust-lang/rust#99217) - [Stringify non-shorthand visibility correctly] (rust-lang/rust#100350) - [Normalize struct field types when unsizing] (rust-lang/rust#101831) - [Update to LLVM 15](rust-lang/rust#99464) - [Fix aarch64 call abi to correctly zeroext when needed] (rust-lang/rust#97800) - [debuginfo: Generalize C++-like encoding for enums] (rust-lang/rust#98393) - [Add `special_module_name` lint] (rust-lang/rust#94467) - [Add support for generating unique profraw files by default when using `-C instrument-coverage`] (rust-lang/rust#100384) - [Allow dynamic linking for iOS/tvOS targets] (rust-lang/rust#100636) New targets: - [Add armv4t-none-eabi as a tier 3 target] (rust-lang/rust#100244) - [Add powerpc64-unknown-openbsd and riscv64-unknown-openbsd as tier 3 targets] (rust-lang/rust#101025) - Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Don't generate `PartialEq::ne` in derive(PartialEq)] (rust-lang/rust#98655) - [Windows RNG: Use `BCRYPT_RNG_ALG_HANDLE` by default] (rust-lang/rust#101325) - [Forbid mixing `System` with direct system allocator calls] (rust-lang/rust#101394) - [Document no support for writing to non-blocking stdio/stderr] (rust-lang/rust#101416) - [`std::layout::Layout` size must not overflow `isize::MAX` when rounded up to `align`](rust-lang/rust#95295) This also changes the safety conditions on `Layout::from_size_align_unchecked`. Stabilized APIs --------------- - [`std::backtrace::Backtrace`] (https://doc.rust-lang.org/stable/std/backtrace/struct.Backtrace.html) - [`Bound::as_ref`] (https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.as_ref) - [`std::io::read_to_string`] (https://doc.rust-lang.org/stable/std/io/fn.read_to_string.html) - [`<*const T>::cast_mut`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_mut) - [`<*mut T>::cast_const`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.cast_const) These APIs are now stable in const contexts: - [`<*const T>::offset_from`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from) - [`<*mut T>::offset_from`] (https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset_from) Cargo ----- - [Apply GitHub fast path even for partial hashes] (rust-lang/cargo#10807) - [Do not add home bin path to PATH if it's already there] (rust-lang/cargo#11023) - [Take priority into account within the pending queue] (rust-lang/cargo#11032). This slightly optimizes job scheduling by Cargo, with typically small improvements on larger crate graph builds. Compatibility Notes ------------------- - [`std::layout::Layout` size must not overflow `isize::MAX` when rounded up to `align`] (rust-lang/rust#95295). This also changes the safety conditions on `Layout::from_size_align_unchecked`. - [`PollFn` now only implements `Unpin` if the closure is `Unpin`] (rust-lang/rust#102737). This is a possible breaking change if users were relying on the blanket unpin implementation. See discussion on the PR for details of why this change was made. - [Drop ExactSizeIterator impl from std::char::EscapeAscii] (rust-lang/rust#99880) This is a backwards-incompatible change to the standard library's surface area, but is unlikely to affect real world usage. - [Do not consider a single repeated lifetime eligible for elision in the return type] (rust-lang/rust#103450) This behavior was unintentionally changed in 1.64.0, and this release reverts that change by making this an error again. - [Reenable disabled early syntax gates as future-incompatibility lints] (rust-lang/rust#99935) - [Update the minimum external LLVM to 13] (rust-lang/rust#100460) - [Don't duplicate file descriptors into stdio fds] (rust-lang/rust#101426) - [Sunset RLS](rust-lang/rust#100863) - [Deny usage of `#![cfg_attr(..., crate_type = ...)]` to set the crate type] (rust-lang/rust#99784) This strengthens the forward compatibility lint deprecated_cfg_attr_crate_type_name to deny. - [`llvm-has-rust-patches` allows setting the build system to treat the LLVM as having Rust-specific patches] (rust-lang/rust#101072) This option may need to be set for distributions that are building Rust with a patched LLVM via `llvm-config`, not the built-in LLVM. Internal Changes ---------------- These changes do not affect any public interfaces of Rust, but they represent significant improvements to the performance or internals of rustc and related tools. - [Add `x.sh` and `x.ps1` shell scripts] (rust-lang/rust#99992) - [compiletest: use target cfg instead of hard-coded tables] (rust-lang/rust#100260) - [Use object instead of LLVM for reading bitcode from rlibs] (rust-lang/rust#98100) - [Enable MIR inlining for optimized compilations] (rust-lang/rust#91743) This provides a 3-10% improvement in compiletimes for real world crates. See [perf results] (https://perf.rust-lang.org/compare.html?start=aedf78e56b2279cc869962feac5153b6ba7001ed&end=0075bb4fad68e64b6d1be06bf2db366c30bc75e1&stat=instructions:u).

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jun 22, 2022

rust-highfive assigned wesleywiser Jun 22, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 22, 2022

michaelwoerister force-pushed the new-cpp-like-enum-debuginfo branch from 8919e1d to 2b51da7 Compare June 22, 2022 16:47

michaelwoerister added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 22, 2022

michaelwoerister mentioned this pull request Jun 30, 2022

Use niche-filling optimization even when multiple variants have data. #94075

Merged

michaelwoerister force-pushed the new-cpp-like-enum-debuginfo branch 4 times, most recently from 2214eed to 64d9011 Compare July 1, 2022 12:03

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 1, 2022

michaelwoerister changed the title ~~[wip] debuginfo: Change C++-like encoding for enums.~~ debuginfo: Generalize C++-like encoding for enums. Jul 1, 2022

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jul 1, 2022

vadimcn reviewed Jul 5, 2022

View reviewed changes

vadimcn reviewed Jul 7, 2022

View reviewed changes

wesleywiser reviewed Jul 7, 2022

View reviewed changes

michaelwoerister force-pushed the new-cpp-like-enum-debuginfo branch from fe42a9e to 02a4928 Compare July 8, 2022 15:24

michaelwoerister removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jul 8, 2022

vadimcn reviewed Jul 8, 2022

View reviewed changes

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 12, 2022

This comment has been minimized.

Sign in to view

[debuginfo] Update src/test/debuginfo/mutex.rs for new cpp-like enum …

6d030a5

…debuginfo encoding.

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 12, 2022

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 12, 2022

wesleywiser reviewed Aug 12, 2022

View reviewed changes

src/test/debuginfo/msvc-pretty-enums.rs Show resolved Hide resolved

src/test/debuginfo/msvc-pretty-enums.rs Show resolved Hide resolved

This comment has been minimized.

Sign in to view

[debuginfo] Fix msvc-pretty-enums debuginfo test for i686.

03b93d0

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 15, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 15, 2022

bors merged commit 4916e2b into rust-lang:master Aug 15, 2022

rustbot added this to the 1.65.0 milestone Aug 15, 2022

MichaelMcDonnell mentioned this pull request Dec 27, 2022

Rust enum printing broken on Windows since Rust 1.65 vadimcn/codelldb#839

Open

matthiaskrgr mentioned this pull request Aug 27, 2023

ICE: no name for HirId #115264

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

debuginfo: Generalize C++-like encoding for enums. #98393

debuginfo: Generalize C++-like encoding for enums. #98393

michaelwoerister commented Jun 22, 2022 •

edited

Loading

michaelwoerister commented Jul 1, 2022

rust-timer commented Jul 1, 2022

bors commented Jul 1, 2022

bors commented Jul 1, 2022

rust-timer commented Jul 1, 2022

rust-timer commented Jul 1, 2022

vadimcn Jul 7, 2022

michaelwoerister Jul 8, 2022

vadimcn Jul 8, 2022

michaelwoerister Jul 20, 2022

vadimcn Jul 7, 2022

michaelwoerister Jul 8, 2022

wesleywiser Jul 7, 2022

michaelwoerister Jul 8, 2022 •

edited

Loading

michaelwoerister commented Jul 8, 2022

michaelwoerister commented Jul 8, 2022

vadimcn Jul 8, 2022

This comment has been minimized.

wesleywiser commented Aug 12, 2022

bors commented Aug 12, 2022

bors commented Aug 12, 2022

bors commented Aug 12, 2022

wesleywiser left a comment

This comment has been minimized.

michaelwoerister commented Aug 15, 2022

bors commented Aug 15, 2022

bors commented Aug 15, 2022

bors commented Aug 15, 2022

rust-timer commented Aug 15, 2022

debuginfo: Generalize C++-like encoding for enums. #98393

debuginfo: Generalize C++-like encoding for enums. #98393

Conversation

michaelwoerister commented Jun 22, 2022 • edited Loading

michaelwoerister commented Jul 1, 2022

rust-timer commented Jul 1, 2022

bors commented Jul 1, 2022

bors commented Jul 1, 2022

rust-timer commented Jul 1, 2022

rust-timer commented Jul 1, 2022

Footnotes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

michaelwoerister Jul 8, 2022 • edited Loading

Choose a reason for hiding this comment

michaelwoerister commented Jul 8, 2022

michaelwoerister commented Jul 8, 2022

Choose a reason for hiding this comment

This comment has been minimized.

wesleywiser commented Aug 12, 2022

bors commented Aug 12, 2022

bors commented Aug 12, 2022

bors commented Aug 12, 2022

wesleywiser left a comment

Choose a reason for hiding this comment

This comment has been minimized.

michaelwoerister commented Aug 15, 2022

bors commented Aug 15, 2022

bors commented Aug 15, 2022

bors commented Aug 15, 2022

rust-timer commented Aug 15, 2022

Footnotes

michaelwoerister commented Jun 22, 2022 •

edited

Loading

michaelwoerister Jul 8, 2022 •

edited

Loading