Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICE: OutputTypeParameterMismatch #117951

Open
dwhjames opened this issue Nov 15, 2023 · 16 comments
Open

ICE: OutputTypeParameterMismatch #117951

dwhjames opened this issue Nov 15, 2023 · 16 comments
Labels
C-bug Category: This is a bug. E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@dwhjames
Copy link

Meta

We have a weekly CI job with a build matrix of Rust versions

  • 1.65.0
  • stable
  • beta
  • nightly

The last passing build was 2023-10-24. The following is a chronology.


2023-10-24
All rust versions PASS ✅

'1.65.0-x86_64-unknown-linux-gnu'
latest update on 2022-11-03, rust version 1.65.0 (897e375 2022-11-02)

'stable-x86_64-unknown-linux-gnu'
latest update on 2023-10-05, rust version 1.73.0 (cc66ad4 2023-10-03)

'beta-x86_64-unknown-linux-gnu'
latest update on 2023-10-21, rust version 1.74.0-beta.4 (489647f 2023-10-21)

'nightly-x86_64-unknown-linux-gnu'
latest update on 2023-10-24, rust version 1.75.0-nightly (cd674d6 2023-10-24)


2023-10-31

PASS for 1.65.0, stable, and beta ✅

'nightly-x86_64-unknown-linux-gnu'
(exact version already GC’ed from CI systems but, reproduced with 2023-10-30 nightly)
FAIL ❌


2023-11-07

PASS for 1.65.0 and stable ✅

'beta-x86_64-unknown-linux-gnu'
latest update on 2023-10-21, rust version 1.74.0-beta.4 (489647f 2023-10-21)
PASS ✅

'nightly-x86_64-unknown-linux-gnu'
latest update on 2023-11-07, rust version 1.75.0-nightly (189d6c7 2023-11-06)
FAIL ❌


2023-11-14

PASS for 1.65.0 and stable ✅

'nightly-x86_64-unknown-linux-gnu'
latest update on 2023-11-14, rust version 1.76.0-nightly (ba7c7a3 2023-11-13)
PASS ✅

'beta-x86_64-unknown-linux-gnu'
latest update on 2023-11-13, rust version 1.75.0-beta.1 (782883f 2023-11-12)
FAIL ❌


Summary:

  • 1.75.0-nightly did succeed on 2023-10-24
  • 1.75.0-nightly was first observed to fail on 2023-10-31
  • 1.76.0-nightly now succeeds (2023-11-14)
  • 1.75.0-beta.1 was first observed to fail on 2023-11-14

Error output

Backtrace

error: internal compiler error: compiler/rustc_traits/src/codegen.rs:43:13: Encountered error `
    OutputTypeParameterMismatch(
      SelectionOutputTypeParameterMismatch {
        found_trait_ref: Binder { value: <fn(<round_digest::RoundDigestTag<<Dps as mpc_signing::protocols::cggmp21::DpsProtocol>::SendAllReport2> as mpc_signing::utils::VerifyDeserialize>::VerifyError) -> ds::VerifyError<Dps, Ds> {ds::VerifyError::<Dps, Ds>::Round2} as std::ops::FnOnce<(<round_digest::RoundDigestTag<<Dps as mpc_signing::protocols::cggmp21::DpsProtocol>::SendAllReport2> as mpc_signing::utils::VerifyDeserialize>::VerifyError,)>>, bound_vars: [] },
        expected_trait_ref: Binder { value: <fn(<round_digest::RoundDigestTag<<Dps as mpc_signing::protocols::cggmp21::DpsProtocol>::SendAllReport2> as mpc_signing::utils::VerifyDeserialize>::VerifyError) -> ds::VerifyError<Dps, Ds> {ds::VerifyError::<Dps, Ds>::Round2} as std::ops::FnOnce<(round_digest::VerifyError<<<Dps as mpc_signing::protocols::cggmp21::DpsProtocol>::SendAllReport2 as mpc_signing::utils::VerifyDeserialize>::VerifyError>,)>>, bound_vars: [] },
        terr: Sorts(ExpectedFound {
          expected: round_digest::VerifyError<Alias(Projection, AliasTy { args: [Alias(Projection, AliasTy { args: [Dps/#0], def_id: DefId(22:4767 ~ mpc_signing[4ddb]::protocols::cggmp21::dps_protocol::traits::DpsProtocol::SendAllReport2) })], def_id: DefId(22:3108 ~ mpc_signing[4ddb]::utils::zkp_stream::VerifyDeserialize::VerifyError) })>,
          found: Alias(Projection, AliasTy { args: [round_digest::RoundDigestTag<Alias(Projection, AliasTy { args: [Dps/#0], def_id: DefId(22:4767 ~ mpc_signing[4ddb]::protocols::cggmp21::dps_protocol::traits::DpsProtocol::SendAllReport2) })>], def_id: DefId(22:3108 ~ mpc_signing[4ddb]::utils::zkp_stream::VerifyDeserialize::VerifyError) }) }) }
    )` selecting `
      <fn(<round_digest::RoundDigestTag<<Dps as mpc_signing::protocols::cggmp21::DpsProtocol>::SendAllReport2> as mpc_signing::utils::VerifyDeserialize>::VerifyError)
        -> ds::VerifyError<Dps, Ds> {ds::VerifyError::<Dps, Ds>::Round2} as std::ops::FnOnce<(round_digest::VerifyError<<<Dps as mpc_signing::protocols::cggmp21::DpsProtocol>::SendAllReport2 as mpc_signing::utils::VerifyDeserialize>::VerifyError>,)>>
  ` during codegen

Capture of Nightly failure (2023-10-31) ❌
rustc-ice-2023-10-31T16_49_05-19815.txt

Capture of Beta failure (2023-11-15) ❌
rustc-ice-2023-11-15.txt

Code

From 2023-10-24 to today the source code has not changed. The code is not public, but could potentially be shared. Unfortunately, the code is complex due to the use of a declarative macro, so we do not currently have a minimization.

We would welcome some hints/suggestions about what code/context we could capture to improve this issue.

@dwhjames dwhjames added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 15, 2023
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Nov 15, 2023
@lqd
Copy link
Member

lqd commented Nov 15, 2023

A bisection using cargo-bisect-rustc (documentation and tutorials are available here; also: use the --access github flag, otherwise it will clone this repo) would at least show the PRs that:

  • caused the error on nightly, and that PR should now be on beta
  • fixed the issue on nightly only

It's likely the PR you will find fixing the issue on nightly will also need to be backported to beta.

@saethlin saethlin added E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc and removed needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Nov 15, 2023
@dwhjames
Copy link
Author

@lqd thanks for the directions. I will take a crack at that.

Could you help me understand the release process… will there be any further promotions of changes in nightly to beta? Or is beta ‘locked in’ and explicit action is required to address this issue in beta?
I am fine waiting for stabilization, but I’m obviously concerned about this issue eventually showing up in stable.

@lqd
Copy link
Member

lqd commented Nov 15, 2023

AFAICT beta has branched already and all further promotions will require explicit decisions and actions. But that’s extremely common and business as usual, there’s an entire section of the weekly compiler meeting dedicated to that.

I could be wrong but I don’t believe crater runs have been done for the next beta yet, and that issue may also show up there (it would if your project was on crates.io for example), but even if not: once more details are known about the severity of the issue, how common it can be, how simple the fix is to backport, and so on, all will be taken into account when these decisions are made.

This will all follow naturally once the PRs are identified.

@dwhjames
Copy link
Author

searched nightlies: from nightly-2023-10-17 to nightly-2023-10-22
regressed nightly: nightly-2023-10-21
searched commit range: 4578435...249624b
regressed commit: 5cee4f3

bisected with cargo-bisect-rustc v0.6.7

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --start=2023-10-17 --end=2023-10-22 --access github --regress=ice --preserve --prompt -- build --release

@dwhjames
Copy link
Author

Had a few false starts here until I figured out the … -- build --release to build in release mode (no ICE in debug mode).

@dwhjames
Copy link
Author

run again with --regress=non-ice (as this failure PASSED in our weekly build on 2023-10-17 and 2023-10-24, so the failure flip flopped during that period)


searched nightlies: from nightly-2023-10-21 to nightly-2023-10-24
fixed nightly: nightly-2023-10-23
searched commit range: 1c05d50...54b0434
fixed commit: 54b0434

bisected with cargo-bisect-rustc v0.6.7

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --start=2023-10-21 --end=2023-10-24 --access github --regress=non-ice --preserve --prompt -- build --release

@dwhjames
Copy link
Author

Further searching 2023-10-24 to 2023-10-31

…
looking for regression commit between 2023-10-25 and 2023-10-26
fetching (via remote github) commits from max(df871fbf053de3a855398964cd05fadbe91cf4fd, 2023-10-23) to ab5c841a1f3c09edc5ea07722519627c960aed17
ending github query because we found starting sha: df871fbf053de3a855398964cd05fadbe91cf4fd
get_commits_between returning commits, len: 13
…

checking the start range to verify it passes
installing df871fbf053de3a855398964cd05fadbe91cf4fd
…

ERROR: the commit at the start of the range (df871fbf053de3a855398964cd05fadbe91cf4fd) includes the regression
bisected with cargo-bisect-rustc v0.6.7

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --start=2023-10-24 --end=2023-10-31 --access github --regress=ice --preserve -- build --release--release

@lqd
Copy link
Member

lqd commented Nov 16, 2023

Kind of unexpected, the PR that fixed the issue is about cargo and shouldn’t change how the compiler is called or behaves. Try double checking maybe, preferably with a script looking for the exact ICE message here rather than with --prompt.

@dwhjames
Copy link
Author

❯ rustup override set nightly-2023-11-14
❯ cargo build --release -p mpc-fsm
💥 

❯ rustup override set nightly-2023-11-15
❯ cargo build --release -p mpc-fsm
✅ 

❯ rustup override unset

Yet

cargo bisect-rustc --start=2023-11-14 --end=2023-11-15 --access github --regress=non-ice --preserve -- build --release

ICE at both start and end 🤔

@dwhjames
Copy link
Author

repeating that with commit hashes

  • nightly-2023-11-14-x86_64-apple-darwin unchanged - rustc 1.76.0-nightly (ba7c7a3 2023-11-13)
  • nightly-2023-11-15-x86_64-apple-darwin unchanged - rustc 1.76.0-nightly (dd430bc 2023-11-14)
cargo bisect-rustc --start=ba7c7a301 --end=dd430bc8c --access github --regress=non-ice --preserve -- build --release -p mpc-fsm

still ICE at both start and end 🤯

@dwhjames
Copy link
Author

reproducing the search for the first regression within a wider date range

searched nightlies: from nightly-2023-10-01 to nightly-2023-10-22
regressed nightly: nightly-2023-10-21
searched commit range: 4578435...249624b
regressed commit: 5cee4f3

bisected with cargo-bisect-rustc v0.6.7

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --start=2023-10-01 --end=2023-10-22 --access github --regress=ice --preserve -- build --release -p mpc-fsm

reproducing the return to non-ICE

searched nightlies: from nightly-2023-10-21 to nightly-2023-10-24
fixed nightly: nightly-2023-10-23
searched commit range: 1c05d50...54b0434
fixed commit: 54b0434

bisected with cargo-bisect-rustc v0.6.7

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --start=2023-10-21 --end=2023-10-24 --access github --regress=non-ice --preserve -- build --release -p mpc-fsm

reproducing the search for the second occurrence of the regression within a wider date range

searched nightlies: from nightly-2023-10-23 to nightly-2023-11-15
regressed nightly: nightly-2023-10-25
searched commit range: f1a5ce1...df871fb
regressed commit: 271dcc1

bisected with cargo-bisect-rustc v0.6.7

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --start=2023-10-23 --end=2023-11-15 --access github --regress=ice --preserve -- build --release -p mpc-fsm

@lqd
Copy link
Member

lqd commented Nov 16, 2023

The latter, #116435, is again a PR that shouldn't have any effect in theory: it's a change to the new trait solver, and that is not used unless a -Ztrait-solver=next flag is passed. Backporting this PR shouldn't fix this issue.

We'd need to verify with a way to reproduce this issue: I'll mark this as needing an MCVE, but we don't need that reproducer to be minimal just yet. People can help reducing it afterwards.

@lqd lqd added the E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example label Nov 16, 2023
@dwhjames
Copy link
Author

An ideas as to why I might see

  • ❌ nightly-2023-11-14-x86_64-apple-darwin unchanged - rustc 1.76.0-nightly (ba7c7a3 2023-11-13)
  • ✅ nightly-2023-11-15-x86_64-apple-darwin unchanged - rustc 1.76.0-nightly (dd430bc 2023-11-14)
  • ❌ nightly-2023-11-16-x86_64-apple-darwin unchanged - rustc 1.76.0-nightly (6b771f6 2023-11-15)

when using a toolchain override with rustup…

yet, I can not reproduce the build passing with cargo-bisect-rustc? (#117951 (comment))

@lqd
Copy link
Member

lqd commented Nov 16, 2023

Not easily without being able to reproduce the issue, unfortunately.

@dwhjames
Copy link
Author

dwhjames commented Dec 14, 2023

I don’t yet have a minimized version of the issue, but I do have new information.

There are several crates in the project. The crash occurs within mpc-fsm and code generated from a large macro. I have gotten inconsistent results when attempting to compile just mpc-fsm crate (cargo build --release -p mpc-fsm). However, I have found when investigating, that compiling a second crate mpc-jni (cargo build --release -p mpc-jni) that depends on mpc-fsm will successfully compile, even when that same compiler version will crash when compiling mpc-fsm in isolation.

Could it be the case that the compilation context would cause some specialization or ‘guidance’, even across crates?

Does anything about this (the macros or the cross crate behavior) help with how I might attempt to minimize this issue (which is daunting given the size)?

The ‘beta’ channel (now at rustc 1.75.0-beta.5 (1a06ac5b5 2023-12-01)) still has the ICE for this project.

The following is a redo of hunting for the original regression, but compiling with the context of the second crate, which gives a different answer compared to #117951 (comment)


searched nightlies: from nightly-2023-10-17 to nightly-2023-10-22
regressed nightly: nightly-2023-10-19
searched commit range: 09df610...0039d73
regressed commit: 862bba6

bisected with cargo-bisect-rustc v0.6.7

Host triple: x86_64-apple-darwin
Reproduce with:

cargo bisect-rustc --start=2023-10-17 --end=2023-10-22 --access github --regress=ice --preserve -- build --release -p mpc-jni

@kpreid
Copy link
Contributor

kpreid commented Dec 25, 2023

Relabeling issues which don't have a public runnable reproduction (as opposed to not having a minimized one) to the new label S-needs-repro.
@rustbot label +S-needs-repro -E-needs-mcve

@rustbot rustbot added S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. and removed E-needs-mcve Call for participation: This issue has a repro, but needs a Minimal Complete and Verifiable Example labels Dec 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. E-needs-bisection Call for participation: This issue needs bisection: https://github.com/rust-lang/cargo-bisect-rustc I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ S-needs-repro Status: This issue has no reproduction and needs a reproduction to make progress. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

5 participants