Fix ICE involving calling `Instance.ty` during const evaluation #67800

Aaron1011 · 2020-01-02T05:51:03Z

Instance.ty assumes that we are in a fully monomorphic context (e.g.
codegen), and can therefore use an empty ParamEnv when performing
normalization. Howver, the MIR constant evaluator code ends up calling
Instance.ty as a result of us attemptign to 'speculatively'
const-evaluate generic functions during const propagation.

As a result,
we may end up with projections involving type parameters
(e.g. ::Bar>) in the type we are trying to normalize.
Normalization expects us to have proper predicates in the ParamEnv for
such projections, and will ICE if we don't.

This commit adds a new method Instance.ty_env, which takes a
ParamEnv for use during normalization. The MIR const-evaluator code is
changed to use this method, passing in the proper ParamEnv for the
context at hand.

rust-highfive · 2020-01-02T05:51:07Z

r? @varkor

(rust_highfive has picked a reviewer for you, use r? to override)

src/librustc/ty/instance.rs

rust-highfive · 2020-01-03T02:00:47Z

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2020-01-03T01:39:32.1959938Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-03T01:39:32.7518807Z ##[command]git config gc.auto 0
2020-01-03T01:39:32.7520920Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-03T01:39:32.7524338Z ##[command]git config --get-all http.proxy
2020-01-03T01:39:32.7527028Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67800/merge:refs/remotes/pull/67800/merge

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

src/librustc/ty/instance.rs

wesleywiser · 2020-01-03T11:23:53Z

@bors try

@rust-timer queue

rust-timer · 2020-01-03T11:23:55Z

Awaiting bors try build completion

bors · 2020-01-03T11:24:07Z

⌛ Trying commit b9dcd9814b4f8b37cfe0b0136cc10ed7797d30eb with merge c8783d8cf524e2ceaad6d300e62e58e68a6d22cf...

bors · 2020-01-03T14:09:48Z

☀️ Try build successful - checks-azure
Build commit: c8783d8cf524e2ceaad6d300e62e58e68a6d22cf (c8783d8cf524e2ceaad6d300e62e58e68a6d22cf)

rust-timer · 2020-01-03T14:09:50Z

Queued c8783d8cf524e2ceaad6d300e62e58e68a6d22cf with parent 4877e16, future comparison URL.

Aaron1011 · 2020-01-03T16:14:49Z

For some reason, the bot didn't comment saying that benchmarking completed.

It looks like this change had no perf impact above the noise floor, as I had hoped.

wesleywiser

Code and perf look good to me and the change seems reasonable enough but I don't think I know enough about the resolver to sign off on this myself. Perhaps @oli-obk or someone else could take a look as well?

src/librustc/ty/instance.rs

Fixes rust-lang#67639 `Instance.ty` assumes that we are in a fully monomorphic context (e.g. codegen), and can therefore use an empty `ParamEnv` when performing normalization. Howver, the MIR constant evaluator code ends up calling `Instance.ty` as a result of us attemptign to 'speculatively' const-evaluate generic functions during const propagation. As a result, we may end up with projections involving type parameters (e.g. <T as MyTrait>::Bar>) in the type we are trying to normalize. Normalization expects us to have proper predicates in the `ParamEnv` for such projections, and will ICE if we don't. This commit adds a new method `Instance.ty_env`, which takes a `ParamEnv` for use during normalization. The MIR const-evaluator code is changed to use this method, passing in the proper `ParamEnv` for the context at hand.

Co-Authored-By: Wesley Wiser <wwiser@gmail.com>

Aaron1011 · 2020-01-05T20:01:20Z

@oli-obk: Updated

src/librustc_mir/interpret/traits.rs

oli-obk · 2020-01-05T22:04:44Z

src/librustc/ty/layout.rs

@@ -2301,7 +2301,7 @@ impl<'tcx> ty::Instance<'tcx> {
    // or should go through `FnAbi` instead, to avoid losing any
    // adjustments `FnAbi::of_instance` might be performing.
    fn fn_sig_for_fn_abi(&self, tcx: TyCtxt<'tcx>) -> ty::PolyFnSig<'tcx> {
-        let ty = self.ty(tcx);
+        let ty = self.monomorphic_ty(tcx);


It's not obvious to me that this is correct. Ca you go up the call chain and see if there are any noncodegen uses?

The only caller is FnAbi:of_instance, which is only called from codegen.

We need to stop assuming that codegen = monomorphic.

oli-obk · 2020-01-05T23:24:00Z

r? @oli-obk

@bors r+

bors · 2020-01-05T23:24:01Z

📌 Commit 336b902 has been approved by oli-obk

… r=oli-obk Fix ICE involving calling `Instance.ty` during const evaluation Fixes rust-lang#67639 `Instance.ty` assumes that we are in a fully monomorphic context (e.g. codegen), and can therefore use an empty `ParamEnv` when performing normalization. Howver, the MIR constant evaluator code ends up calling `Instance.ty` as a result of us attemptign to 'speculatively' const-evaluate generic functions during const propagation. As a result, we may end up with projections involving type parameters (e.g. <T as MyTrait>::Bar>) in the type we are trying to normalize. Normalization expects us to have proper predicates in the `ParamEnv` for such projections, and will ICE if we don't. This commit adds a new method `Instance.ty_env`, which takes a `ParamEnv` for use during normalization. The MIR const-evaluator code is changed to use this method, passing in the proper `ParamEnv` for the context at hand.

@ghost

Rollup of 6 pull requests Successful merges: - #67800 (Fix ICE involving calling `Instance.ty` during const evaluation) - #67873 (change remove to have a PartialEq bound) - #67897 (Use `as_deref()` to replace `as_ref().map(...)`) - #67906 (Silence `TooGeneric` error) - #67912 (macros: typo fix) - #67915 (Use Self instead of $type) Failed merges: r? @ghost

eddyb · 2020-03-02T22:18:28Z

src/librustc/ty/instance.rs

-    pub fn ty(&self, tcx: TyCtxt<'tcx>) -> Ty<'tcx> {
+    /// Returns the `Ty` corresponding to this `Instance`,
+    /// with generic substitutions applied and lifetimes erased.
+    ///
+    /// This method can only be called when the 'substs' for this Instance
+    /// are fully monomorphic (no `ty::Param`'s are present).
+    /// This is usually the case (e.g. during codegen).
+    /// However, during constant evaluation, we may want
+    /// to try to resolve a `Instance` using generic parameters
+    /// (e.g. when we are attempting to to do const-propagation).
+    /// In this case, `Instance.ty_env` should be used to provide
+    /// the `ParamEnv` for our generic context.
+    pub fn monomorphic_ty(&self, tcx: TyCtxt<'tcx>) -> Ty<'tcx> {
        let ty = tcx.type_of(self.def.def_id());
+        // There shouldn't be any params - if there are, then
+        // Instance.ty_env should have been used to provide the proper
+        // ParamEnv
+        if self.substs.has_param_types() {
+            bug!("Instance.ty called for type {:?} with params in substs: {:?}", ty, self.substs);
+        }
        tcx.subst_and_normalize_erasing_regions(self.substs, ty::ParamEnv::reveal_all(), &ty)
    }
+
+    /// Like `Instance.ty`, but allows a `ParamEnv` to be specified for use during
+    /// normalization. This method is only really useful during constant evaluation,
+    /// where we are dealing with potentially generic types.
+    pub fn ty_env(&self, tcx: TyCtxt<'tcx>, param_env: ty::ParamEnv<'tcx>) -> Ty<'tcx> {
+        let ty = tcx.type_of(self.def.def_id());
+        tcx.subst_and_normalize_erasing_regions(self.substs, param_env, &ty)
+    }


I would keep a single ty method and force callers to pass in ty::ParamEnv::reveal_all() (eventually all of codegen should have to keep track of a ParamEnv anyway, because of @davidtwco's work on partial monomorphization).

rust-highfive assigned varkor Jan 2, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 2, 2020

Aaron1011 changed the title ~~Fix ICE involving calling Instance.ty durign const evaluation~~ Fix ICE involving calling Instance.ty during const evaluation Jan 2, 2020

Aaron1011 force-pushed the fix/mir-generic-instance branch from 51b9fa5 to 1e08f9e Compare January 2, 2020 05:51

This comment has been minimized.

Sign in to view

Aaron1011 force-pushed the fix/mir-generic-instance branch from 1e08f9e to 7e86561 Compare January 2, 2020 09:46

This comment has been minimized.

Sign in to view

wesleywiser requested changes Jan 2, 2020

View reviewed changes

src/librustc/ty/instance.rs Outdated Show resolved Hide resolved

Aaron1011 mentioned this pull request Jan 3, 2020

ICE w/ const_generics when trying to use the size of an associated type size as an array length #67739

Closed

Aaron1011 commented Jan 3, 2020

View reviewed changes

src/librustc/ty/instance.rs Show resolved Hide resolved

wesleywiser approved these changes Jan 3, 2020

View reviewed changes

oli-obk reviewed Jan 5, 2020

View reviewed changes

src/librustc/ty/instance.rs Outdated Show resolved Hide resolved

bjorn3 reviewed Jan 5, 2020

View reviewed changes

src/librustc/ty/instance.rs Outdated Show resolved Hide resolved

Aaron1011 and others added 5 commits January 5, 2020 14:37

Change 'panic!' to 'bug!'

d41f9dd

Co-Authored-By: Wesley Wiser <wwiser@gmail.com>

Run rustfmt

71d163b

s/projections/params/

464b58c

Rename Instance.ty to Instance.monomorphic_ty

33caf0b

Aaron1011 force-pushed the fix/mir-generic-instance branch from b9dcd98 to 33caf0b Compare January 5, 2020 20:00

Fix typo

db2c4f2

oli-obk reviewed Jan 5, 2020

View reviewed changes

src/librustc_mir/interpret/traits.rs Outdated Show resolved Hide resolved

oli-obk reviewed Jan 5, 2020

View reviewed changes

Use Instance.ty_env instead of Instance.monomorphic_ty in interpreter

336b902

rust-highfive assigned oli-obk and unassigned varkor Jan 5, 2020

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 5, 2020

Dylan-DPC-zz mentioned this pull request Jan 6, 2020

Rollup of 6 pull requests #67917

Merged

bors merged commit 336b902 into rust-lang:master Jan 6, 2020

eddyb reviewed Mar 2, 2020

View reviewed changes

This was referenced Mar 6, 2020

Unwanted upstream degradation from this app. wei/pull#185

Closed

Polymorphization #69749

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix ICE involving calling `Instance.ty` during const evaluation #67800

Fix ICE involving calling `Instance.ty` during const evaluation #67800

Aaron1011 commented Jan 2, 2020

rust-highfive commented Jan 2, 2020

This comment has been minimized.

This comment has been minimized.

rust-highfive commented Jan 3, 2020

wesleywiser commented Jan 3, 2020

rust-timer commented Jan 3, 2020

bors commented Jan 3, 2020

bors commented Jan 3, 2020

rust-timer commented Jan 3, 2020

Aaron1011 commented Jan 3, 2020

wesleywiser left a comment

Aaron1011 commented Jan 5, 2020

oli-obk Jan 5, 2020

Aaron1011 Jan 5, 2020

eddyb Mar 2, 2020

oli-obk commented Jan 5, 2020

bors commented Jan 5, 2020

eddyb Mar 2, 2020

Fix ICE involving calling Instance.ty during const evaluation #67800

Fix ICE involving calling Instance.ty during const evaluation #67800

Conversation

Aaron1011 commented Jan 2, 2020

rust-highfive commented Jan 2, 2020

This comment has been minimized.

This comment has been minimized.

rust-highfive commented Jan 3, 2020

wesleywiser commented Jan 3, 2020

rust-timer commented Jan 3, 2020

bors commented Jan 3, 2020

bors commented Jan 3, 2020

rust-timer commented Jan 3, 2020

Aaron1011 commented Jan 3, 2020

wesleywiser left a comment

Choose a reason for hiding this comment

Aaron1011 commented Jan 5, 2020

oli-obk Jan 5, 2020

Choose a reason for hiding this comment

Aaron1011 Jan 5, 2020

Choose a reason for hiding this comment

eddyb Mar 2, 2020

Choose a reason for hiding this comment

oli-obk commented Jan 5, 2020

bors commented Jan 5, 2020

eddyb Mar 2, 2020

Choose a reason for hiding this comment

Fix ICE involving calling `Instance.ty` during const evaluation #67800

Fix ICE involving calling `Instance.ty` during const evaluation #67800