Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Rust Symbol Mangling by default #85530

Closed
wants to merge 6 commits into from

Conversation

tmiasko
Copy link
Contributor

@tmiasko tmiasko commented May 20, 2021

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 20, 2021
@tmiasko
Copy link
Contributor Author

tmiasko commented May 20, 2021

@bors try

@bors
Copy link
Contributor

bors commented May 20, 2021

⌛ Trying commit 493a43f55df7482188e83d68b8ca1bfc26bfd11a with merge ce59db7a1b1b012fb2793c4641c1bdecad7a128b...

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented May 20, 2021

☀️ Try build successful - checks-actions
Build commit: ce59db7a1b1b012fb2793c4641c1bdecad7a128b (ce59db7a1b1b012fb2793c4641c1bdecad7a128b)

@tmiasko
Copy link
Contributor Author

tmiasko commented May 21, 2021

@craterbot run mode=build-and-test

@craterbot
Copy link
Collaborator

🔒 Error: you're not allowed to interact with this bot.

🔑 If you are a member of the Rust team and need access, add yourself to the whitelist.
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@michaelwoerister
Copy link
Member

@craterbot run mode=build-and-test

@craterbot
Copy link
Collaborator

👌 Experiment pr-85530 created and queued.
🤖 Automatically detected try build ce59db7a1b1b012fb2793c4641c1bdecad7a128b
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 21, 2021
@michaelwoerister
Copy link
Member

Thanks for the PR, @tmiasko!

cc #60705

@bjorn3
Copy link
Member

bjorn3 commented May 24, 2021

Should this get a perf run?

@klensy
Copy link
Contributor

klensy commented May 24, 2021

Should this get a perf run?

As i understand, this pr does double work: it mangle and demangle at the same time

 if let Err(_) = rustc_demangle::try_demangle(&mangled) {
                bug!("demangle error: {:?}\n{}", instance, mangled);
            }

So perf run will show some non related info.

@michaelwoerister
Copy link
Member

Yeah, I was debating whether it makes sense to do a perf run here but as @klensy points out, we also do the demangling check.

I would be interested to see how expensive that demangling check is. But we'll need a compiler with the new mangling scheme enabled by default as baseline. We could do that in another PR. It would be great if we could do the demangling check unconditionally -- although doing it as a debug assertion would also be a big improvement.

@Mark-Simulacrum
Copy link
Member

@michaelwoerister @tmiasko Can you share why this needs to be a build-and-test Crater run? I think just building should suffice; that builds tests too: https://github.com/rust-lang/crater/blob/aa40b144e61f7f1c6ba6a3bfc5e85b52b9e4a177/src/runner/test.rs#L233

If mangling changes introduce different runtime behavior that would seem very surprising to me and likely hard to catch reliably with crater (e.g., collisions seem likely to be pretty hard to identify unless they cause obvious linker errors).

@tmiasko
Copy link
Contributor Author

tmiasko commented May 30, 2021

@Mark-Simulacrum potential interaction with mechanisms for collecting & formatting backtraces was one of motivations for opting for some runtime coverage. Generally, reducing the scope of a crater run to build only, seems fine to me.

@craterbot
Copy link
Collaborator

🚧 Experiment pr-85530 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment pr-85530 is completed!
📊 78 regressed and 58 fixed (163710 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jun 9, 2021
@bjorn3
Copy link
Member

bjorn3 commented Jun 9, 2021

ICE for encoding_rs-0.8.28:

[INFO] [stderr] thread 'rustc' panicked at 'Box<Any>', /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/panic.rs:59:5
[INFO] [stdout] error: internal compiler error: compiler/rustc_symbol_mangling/src/v0.rs:568:17: symbol_names: unsupported constant of type `[u32; 16]` (Const { ty: [u32; 16], val: Value(ByRef { alloc: Allocation { bytes: [8, 0, 0, 0, 24, 0, 0, 0, 9, 0, 0, 0, 25, 0, 0, 0, 10, 0, 0, 0, 26, 0, 0, 0, 11, 0, 0, 0, 27, 0, 0, 0, 12, 0, 0, 0, 28, 0, 0, 0, 13, 0, 0, 0, 29, 0, 0, 0, 14, 0, 0, 0, 30, 0, 0, 0, 15, 0, 0, 0, 31, 0, 0, 0], relocations: Relocations(SortedMap { data: [] }), init_mask: InitMask { blocks: [18446744073709551615, 0], len: Size { raw: 64 } }, align: Align { pow2: 2 }, mutability: Not, extra: () }, offset: Size { raw: 0 } }) })
[INFO] [stdout] 
[INFO] [stdout] 
[INFO] [stderr] stack backtrace:
[INFO] [stderr]    0:     0x7f328cbf74c0 - std[77c0456e0eb29f0c]::backtrace_rs::backtrace::libunwind::trace
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
[INFO] [stderr]    1:     0x7f328cbf74c0 - std[77c0456e0eb29f0c]::backtrace_rs::backtrace::trace_unsynchronized::<std[77c0456e0eb29f0c]::sys_common::backtrace::_print_fmt::{closure#1}>
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
[INFO] [stderr]    2:     0x7f328cbf74c0 - std[77c0456e0eb29f0c]::sys_common::backtrace::_print_fmt
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/sys_common/backtrace.rs:67:5
[INFO] [stderr]    3:     0x7f328cbf74c0 - <std[77c0456e0eb29f0c]::sys_common::backtrace::_print::DisplayBacktrace as core[ee1ca6415511a9c4]::fmt::Display>::fmt
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/sys_common/backtrace.rs:46:22
[INFO] [stderr]    4:     0x7f328cc64e9c - core[ee1ca6415511a9c4]::fmt::write
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/core/src/fmt/mod.rs:1110:17
[INFO] [stderr]    5:     0x7f328cbe8e15 - <std[77c0456e0eb29f0c]::sys::unix::stdio::Stderr as std[77c0456e0eb29f0c]::io::Write>::write_fmt
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/io/mod.rs:1584:15
[INFO] [stderr]    6:     0x7f328cbfb20b - std[77c0456e0eb29f0c]::sys_common::backtrace::_print
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/sys_common/backtrace.rs:49:5
[INFO] [stderr]    7:     0x7f328cbfb20b - std[77c0456e0eb29f0c]::sys_common::backtrace::print
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/sys_common/backtrace.rs:36:9
[INFO] [stderr]    8:     0x7f328cbfb20b - std[77c0456e0eb29f0c]::panicking::default_hook::{closure#1}
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/panicking.rs:208:50
[INFO] [stderr]    9:     0x7f328cbface1 - std[77c0456e0eb29f0c]::panicking::default_hook
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/panicking.rs:225:9
[INFO] [stderr]   10:     0x7f328d3b0afd - rustc_driver[483abbcf9e72e7cf]::report_ice
[INFO] [stderr]   11:     0x7f328cbfba16 - std[77c0456e0eb29f0c]::panicking::rust_panic_with_hook
[INFO] [stderr]                                at /rustc/ce59db7a1b1b012fb2793c4641c1bdecad7a128b/library/std/src/panicking.rs:626:17
[INFO] [stderr]   12:     0x7f328e37234b - std[77c0456e0eb29f0c]::panicking::begin_panic::<rustc_errors[a6601eba5631ab8e]::ExplicitBug>::{closure#0}
[INFO] [stderr]   13:     0x7f328e372296 - std[77c0456e0eb29f0c]::sys_common::backtrace::__rust_end_short_backtrace::<std[77c0456e0eb29f0c]::panicking::begin_panic<rustc_errors[a6601eba5631ab8e]::ExplicitBug>::{closure#0}, !>
[INFO] [stderr]   14:     0x7f328e37361f - std[77c0456e0eb29f0c]::panicking::begin_panic::<rustc_errors[a6601eba5631ab8e]::ExplicitBug>
[INFO] [stderr]   15:     0x7f328e38b73d - std[77c0456e0eb29f0c]::panic::panic_any::<rustc_errors[a6601eba5631ab8e]::ExplicitBug>
[INFO] [stderr]   16:     0x7f328e38e5da - <rustc_errors[a6601eba5631ab8e]::HandlerInner>::bug
[INFO] [stderr]   17:     0x7f328e38e080 - <rustc_errors[a6601eba5631ab8e]::Handler>::bug
[INFO] [stderr]   18:     0x7f328e2c8450 - rustc_middle[4cd8eb4e66530e50]::ty::context::tls::with_opt::<rustc_middle[4cd8eb4e66530e50]::util::bug::opt_span_bug_fmt<rustc_span[41f5d8395d191526]::span_encoding::Span>::{closure#0}, ()>
[INFO] [stderr]   19:     0x7f328e2cd500 - rustc_middle[4cd8eb4e66530e50]::util::bug::opt_span_bug_fmt::<rustc_span[41f5d8395d191526]::span_encoding::Span>
[INFO] [stderr]   20:     0x7f328e2cd476 - rustc_middle[4cd8eb4e66530e50]::util::bug::bug_fmt
[INFO] [stderr]   21:     0x7f328f4f7957 - <rustc_symbol_mangling[e3ac7cad7aae7613]::v0::SymbolMangler as rustc_middle[4cd8eb4e66530e50]::ty::print::Printer>::print_const
[INFO] [stderr]   22:     0x7f328ead1f79 - <rustc_symbol_mangling[e3ac7cad7aae7613]::v0::SymbolMangler as rustc_middle[4cd8eb4e66530e50]::ty::print::Printer>::print_def_path
[INFO] [stderr]   23:     0x7f328eace1ac - rustc_symbol_mangling[e3ac7cad7aae7613]::v0::mangle
[INFO] [stderr]   24:     0x7f328eadafba - rustc_symbol_mangling[e3ac7cad7aae7613]::symbol_name_provider
[INFO] [stderr]   25:     0x7f328e7a7183 - <rustc_query_impl[73652419363b7eb3]::queries::symbol_name as rustc_query_system[195a98efe178d20a]::query::config::QueryAccessors<rustc_query_impl[73652419363b7eb3]::plumbing::QueryCtxt>>::compute
[INFO] [stderr]   26:     0x7f328e7ee766 - <rustc_query_system[195a98efe178d20a]::dep_graph::graph::DepGraph<rustc_middle[4cd8eb4e66530e50]::dep_graph::dep_node::DepKind>>::with_task_impl::<rustc_query_impl[73652419363b7eb3]::plumbing::QueryCtxt, rustc_middle[4cd8eb4e66530e50]::ty::instance::Instance, rustc_middle[4cd8eb4e66530e50]::ty::SymbolName, for<'a, 'b> fn(&'a mut rustc_middle[4cd8eb4e66530e50]::ich::hcx::StableHashingContext, &'b rustc_middle[4cd8eb4e66530e50]::ty::SymbolName) -> core[ee1ca6415511a9c4]::option::Option<rustc_data_structures[52e6d6087bd5b52c]::fingerprint::Fingerprint>>
[INFO] [stderr]   27:     0x7f328e7fa9af - rustc_data_structures[52e6d6087bd5b52c]::stack::ensure_sufficient_stack::<(rustc_middle[4cd8eb4e66530e50]::ty::SymbolName, rustc_query_system[195a98efe178d20a]::dep_graph::graph::DepNodeIndex), rustc_query_system[195a98efe178d20a]::query::plumbing::force_query_with_job<rustc_query_system[195a98efe178d20a]::query::caches::DefaultCache<rustc_middle[4cd8eb4e66530e50]::ty::instance::Instance, rustc_middle[4cd8eb4e66530e50]::ty::SymbolName>, rustc_query_impl[73652419363b7eb3]::plumbing::QueryCtxt>::{closure#0}::{closure#0}>
[...]

@tmiasko
Copy link
Contributor Author

tmiasko commented Jun 9, 2021

Errors related to the use of unstable const generics feature with types that are currently unsupported in v0 mangling:

  • ametisf/snapshot_repeat <impl at src/lib.rs:30:1: 40:2>::to_norm (unsupported constant of type f32)
  • bounded 0.1.2 temp::temp::<std::collections::Bound::<usize>::Included(0_usize), std::collections::Bound::<usize>::Included(9_usize)>
  • danielbank/rust-const-generics <impl at examples/state.rs:17:1: 25:2>::freeze (unsupported constant of type &str)
  • encoding-rs 0.8.28 packed_simd::__shuffle_vector16::<[8_u32, 24_u32, 9_u32, 25_u32, 10_u32, 26_u32, 11_u32, 27_u32, 12_u32, 28_u32, 13_u32, 29_u32, 14_u32, 30_u32, 15_u32, 31_u32], packed_simd::codegen::v128::u8x16, packed_simd::codegen::v128::u8x16>
  • fast_rsync 0.1.3 packed_simd_2::__shuffle_vector8::<[2_u32, 10_u32, 3_u32, 11_u32, 6_u32, 14_u32, 7_u32, 15_u32], packed_simd_2::codegen::v256::u32x8, packed_simd_2::codegen::v256::u32x8>
  • nommy 0.4.1 parse::<[text::tag::Tag<".">; 3], std::str::Chars>
  • ritobanrc/firework <typetag::erased_serde::de::erase::Visitor<objects::rect::_::<impl at src/objects/rect.rs:13:28: 13:46>::deserialize::__FieldVisitor> as typetag::erased_serde::de::Visitor>::erased_visit_i64 (unsupported constant of type Axis)
  • tiny-uom 0.1.0 <tiny_uom::Quantity<tiny_uom::Unit { m: 1_i8, kg: 0_i8, s: -1_i8, A: 0_i8, K: 0_i8, mol: 0_i8, cd: 0_i8 }> as std::fmt::Debug>::fmt
  • umbra-lang 0.15.0 ops::<impl at src/ops.rs:637:1: 654:2>::op (unsupported constant of type ops::Op)

In logs there are two dozens of backtraces with mangled symbols (in cases where crate internally uses rustc-demangle crate without support for v0 mangling scheme).

The backtraces generated for RUST_BACKTRACE=full now include a crate hash for every mangled path, so there might be multiple hashes per symbol. Together with generic arguments this is even more verbose than before.

Summary of results:

Result legacy v0
broken:cargo-toml 1912 1912
broken:missing-deps 2730 2730
broken:missing-git-repository 12747 12747
broken:yanked 377 377
build-fail:compiler-error 9201 9172
build-fail:depends-on 13584 13551
build-fail:ice 3 10
build-fail:oom 7201 7460
build-fail:timeout 78 72
build-fail:unknown 10728 10728
error 328 328
test-fail:oom 599 625
test-fail:timeout 277 272
test-fail:unknown 8730 8698
test-pass 95149 94962
test-skipped 59 59

The most immediate question raised by those results is related to const generics. Enabling new mangling scheme will make it harder to experiment with unstable parts of const generics. Should we wait with switching the default until new mangling reaches parity with legacy scheme?

@tmiasko
Copy link
Contributor Author

tmiasko commented Jun 18, 2021

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 18, 2021
@eddyb
Copy link
Member

eddyb commented Jul 18, 2021

As @tmiasko rebased this PR on #87194, I'll try rerunning crater (only on the regressed list), with the combined changes.

@bors try

@bors
Copy link
Contributor

bors commented Jul 18, 2021

⌛ Trying commit 083d4a3 with merge 2b00690a2a9987274f9d399383a581d5df1bf5df...

@bors
Copy link
Contributor

bors commented Jul 18, 2021

☀️ Try build successful - checks-actions
Build commit: 2b00690a2a9987274f9d399383a581d5df1bf5df (2b00690a2a9987274f9d399383a581d5df1bf5df)

@eddyb
Copy link
Member

eddyb commented Jul 18, 2021

@craterbot
Copy link
Collaborator

🚨 Error: failed to parse the command

🆘 If you have any trouble with Crater please ping @rust-lang/infra!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@eddyb
Copy link
Member

eddyb commented Jul 18, 2021

@craterbot
Copy link
Collaborator

🚨 Error: missing desired crates: {"https://crater-reports.s3.amazonaws.com/pr-85530/retry-regressed-list.txt"}

🆘 If you have any trouble with Crater please ping @rust-lang/infra!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@eddyb
Copy link
Member

eddyb commented Jul 18, 2021

@craterbot run mode=build-and-test crates=https://crater-reports.s3.amazonaws.com/pr-85530/retry-regressed-list.txt p=1

@Mark-Simulacrum if this one works (see my previous failed attempt above), the crater docs need updating AFAICT.

@craterbot
Copy link
Collaborator

👌 Experiment pr-85530-1 created and queued.
🤖 Automatically detected try build 2b00690a2a9987274f9d399383a581d5df1bf5df
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 18, 2021
@craterbot
Copy link
Collaborator

🚧 Experiment pr-85530-1 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment pr-85530-1 is completed!
📊 15 regressed and 11 fixed (78 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jul 20, 2021
@eddyb
Copy link
Member

eddyb commented Jul 20, 2021

I'm not sure I'm not missing anything (I didn't open every single log), but AFAICT this is the only regression now:

error: internal compiler error: compiler/rustc_symbol_mangling/src/v0.rs:693:17:
  symbol_names: unsupported constant of type `f32` (Const { ty: f32, val: Value(Scalar(0x3f800000)) })

I explicitly said I don't want to support f32, it's a bug that it's not disallowed much earlier in the compilation, so this is fine.

@tmiasko
Copy link
Contributor Author

tmiasko commented Aug 8, 2021

It looks like #87194 will contain all necessary changes, and this PR already served its role for crater run, so I am going to close it now.

Remaining test changes, that are not already part of #87194, landed in #87789.

@tmiasko tmiasko closed this Aug 8, 2021
@tmiasko tmiasko deleted the rust-symbol-mangling branch August 8, 2021 10:45
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 26, 2021
…woerister,oli-obk

rustc_symbol_mangling: support structural constants and &str in v0.

This PR should unblock rust-lang#85530 (except for float `const` generics, which AFAIK should've never worked).
(cc `@tmiasko` could the rust-lang#85530 (comment) failures be retried with a quick crater "subset" run of this PR + changing the default to `v0`? Just to make sure I didn't miss anything other than the floats)

The encoding is the one suggested before in e.g. rust-lang#61486 (comment), tho this PR won't by itself finish rust-lang#61486, before closing that we'd likely want to move to `@oli-obk's` "valtrees" (i.e. rust-lang#83234 and other associated work).

<hr>

**EDITs**:
1. switched unit/tuple/braced-with-named-fields `<const-fields>` prefixes from `"u"`/`"T"`/`""` to `"U"`/`"T"`/`"S"` to avoid the ambiguity reported by `@tmiasko` in rust-lang#87194 (comment).

2. `rustc-demangle` PR: rust-lang/rustc-demangle#55

3. RFC amendment PR: rust-lang/rfcs#3161
    * also removed the grammar changes included in that PR, from this description

4. added tests (temporarily using my fork of `rustc-demangle`)

<hr>

r? `@michaelwoerister`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.