new solver: make all goal evaluation able to be automatically rerun #108896

BoxyUwU · 2023-03-08T11:02:12Z

It is generally wrong to call evaluate_goal multiple times or evaluate_goal and evaluate_all for the same QueryResult without correctly handling rerunning the goals when inference makes progress. Not doing so will result in the assertion in evaluate_goal firing because rerunning the goal will lead to a more accurate QueryResult.

Currently there are lots of places that get this wrong and generally it is complex and error prone to handle correctly everywhere. This PR introduces a way to add goals to the EvalCtxt and then run all the added goals in a loop so that evaluate_goal/evaluate_all is not necessary to call manually.

There are a few complications for making everything work "right":

the normalizes-to hack that replaces the rhs with an unconstrained infer var requires special casing in the new try_evaluate_added_goals function similar to how evaluate_goal's assertion special cases that hack.
assemble_candidates_after_normalizing_self_ty's normalization step needs to be reran for each candidate otherwise the found candidates will potentially get a more accurate QueryResult when rerunning the projection/trait goal which can effect the QueryResult of the projection/trait goal.
This is implemented via EvalCtxt::probe's closure's EvalCtxt inheriting the added goals of the EvalCtxt that probe is called on, allowing us to add goals in a probe, and then enter a nested probe for each candidate and evaluate added goals which include the normalization step's goals.

I made make_canonical_response evaluate added goals so that it will be hard to mess up the impl of the solver by forgetting to evaluate added goals. Right now the only way to mess this up would be to call response_no_constraints (which from the name is obviously weird).

The visibility of evaluate_goal means that it can be called from various compute_x_goal or candidate assembly functions, this is generally wrong and we should never call evaluate_goal manually, instead we should be calling add_goal/add_goals. This is solved by moving evaluate_goal evaluate_canonical_goal and compute_goal into eval_ctxt's module and making them private so they cannot be called from elsewhere, forcing people to call add_goal/s and evaluate_added_goals_and_make_canonical_resposne/try_evaluate_added_goals

Other changes:

removed the && false that was introduced to the assertion in evaluate_goal in Canonicalize root var when making response from new solver #108839
remove a !self.did_overflow() requirement in search_graph.is_empty() which causes goals that overflow to ICE
made EvalCtxt::eq take &mut self and add all the nested goals via add_goals instead of returning them as 99% of call sites just immediately called EvalCtxt::add_goals manually.

r? @lcnr

rustbot · 2023-03-08T11:02:21Z

Some changes occurred to the core trait solver

cc @rust-lang/initiative-trait-system-refactor

bors · 2023-03-11T22:04:22Z

☔ The latest upstream changes (presumably #109019) made this pull request unmergeable. Please resolve the merge conflicts.

lcnr

happy with this code, some final (hopefully) nits and ideas

compiler/rustc_trait_selection/src/solve/eval_ctxt.rs

compiler/rustc_trait_selection/src/solve/mod.rs

Co-authored-by: lcnr <rust@lcnr.de>

compiler/rustc_trait_selection/src/solve/mod.rs

compiler/rustc_trait_selection/src/solve/trait_goals.rs

lcnr · 2023-03-20T09:01:10Z

@bors r+ rollup

bors · 2023-03-20T09:01:13Z

📌 Commit b85bc19 has been approved by lcnr

It is now in the queue for this repository.

…lcnr new solver: make all goal evaluation able to be automatically rerun It is generally wrong to call `evaluate_goal` multiple times or `evaluate_goal` and `evaluate_all` for the same `QueryResult` without correctly handling rerunning the goals when inference makes progress. Not doing so will result in the assertion in `evaluate_goal` firing because rerunning the goal will lead to a more accurate `QueryResult`. Currently there are lots of places that get this wrong and generally it is complex and error prone to handle correctly everywhere. This PR introduces a way to add goals to the `EvalCtxt` and then run all the added goals in a loop so that `evaluate_goal`/`evaluate_all` is not necessary to call manually. There are a few complications for making everything work "right": 1. the `normalizes-to` hack that replaces the rhs with an unconstrained infer var requires special casing in the new `try_evaluate_added_goals` function similar to how `evaluate_goal`'s assertion special cases that hack. 2. `assemble_candidates_after_normalizing_self_ty`'s normalization step needs to be reran for each candidate otherwise the found candidates will potentially get a more accurate `QueryResult` when rerunning the projection/trait goal which can effect the `QueryResult` of the projection/trait goal. This is implemented via `EvalCtxt::probe`'s closure's `EvalCtxt` inheriting the added goals of the `EvalCtxt` that `probe` is called on, allowing us to add goals in a probe, and then enter a nested probe for each candidate and evaluate added goals which include the normalization step's goals. I made `make_canonical_response` evaluate added goals so that it will be hard to mess up the impl of the solver by forgetting to evaluate added goals. Right now the only way to mess this up would be to call `response_no_constraints` (which from the name is obviously weird). The visibility of `evaluate_goal` means that it can be called from various `compute_x_goal` or candidate assembly functions, this is generally wrong and we should never call `evaluate_goal` manually, instead we should be calling `add_goal`/`add_goals`. This is solved by moving `evaluate_goal` `evaluate_canonical_goal` and `compute_goal` into `eval_ctxt`'s module and making them private so they cannot be called from elsewhere, forcing people to call `add_goal/s` and `evaluate_added_goals_and_make_canonical_resposne`/`try_evaluate_added_goals` --- Other changes: - removed the `&& false` that was introduced to the assertion in `evaluate_goal` in rust-lang#108839 - remove a `!self.did_overflow()` requirement in `search_graph.is_empty()` which causes goals that overflow to ICE - made `EvalCtxt::eq` take `&mut self` and add all the nested goals via `add_goals` instead of returning them as 99% of call sites just immediately called `EvalCtxt::add_goals` manually. r? `@lcnr`

…lcnr new solver: make all goal evaluation able to be automatically rerun It is generally wrong to call `evaluate_goal` multiple times or `evaluate_goal` and `evaluate_all` for the same `QueryResult` without correctly handling rerunning the goals when inference makes progress. Not doing so will result in the assertion in `evaluate_goal` firing because rerunning the goal will lead to a more accurate `QueryResult`. Currently there are lots of places that get this wrong and generally it is complex and error prone to handle correctly everywhere. This PR introduces a way to add goals to the `EvalCtxt` and then run all the added goals in a loop so that `evaluate_goal`/`evaluate_all` is not necessary to call manually. There are a few complications for making everything work "right": 1. the `normalizes-to` hack that replaces the rhs with an unconstrained infer var requires special casing in the new `try_evaluate_added_goals` function similar to how `evaluate_goal`'s assertion special cases that hack. 2. `assemble_candidates_after_normalizing_self_ty`'s normalization step needs to be reran for each candidate otherwise the found candidates will potentially get a more accurate `QueryResult` when rerunning the projection/trait goal which can effect the `QueryResult` of the projection/trait goal. This is implemented via `EvalCtxt::probe`'s closure's `EvalCtxt` inheriting the added goals of the `EvalCtxt` that `probe` is called on, allowing us to add goals in a probe, and then enter a nested probe for each candidate and evaluate added goals which include the normalization step's goals. I made `make_canonical_response` evaluate added goals so that it will be hard to mess up the impl of the solver by forgetting to evaluate added goals. Right now the only way to mess this up would be to call `response_no_constraints` (which from the name is obviously weird). The visibility of `evaluate_goal` means that it can be called from various `compute_x_goal` or candidate assembly functions, this is generally wrong and we should never call `evaluate_goal` manually, instead we should be calling `add_goal`/`add_goals`. This is solved by moving `evaluate_goal` `evaluate_canonical_goal` and `compute_goal` into `eval_ctxt`'s module and making them private so they cannot be called from elsewhere, forcing people to call `add_goal/s` and `evaluate_added_goals_and_make_canonical_resposne`/`try_evaluate_added_goals` --- Other changes: - removed the `&& false` that was introduced to the assertion in `evaluate_goal` in rust-lang#108839 - remove a `!self.did_overflow()` requirement in `search_graph.is_empty()` which causes goals that overflow to ICE - made `EvalCtxt::eq` take `&mut self` and add all the nested goals via `add_goals` instead of returning them as 99% of call sites just immediately called `EvalCtxt::add_goals` manually. r? ``@lcnr``

…lcnr new solver: make all goal evaluation able to be automatically rerun It is generally wrong to call `evaluate_goal` multiple times or `evaluate_goal` and `evaluate_all` for the same `QueryResult` without correctly handling rerunning the goals when inference makes progress. Not doing so will result in the assertion in `evaluate_goal` firing because rerunning the goal will lead to a more accurate `QueryResult`. Currently there are lots of places that get this wrong and generally it is complex and error prone to handle correctly everywhere. This PR introduces a way to add goals to the `EvalCtxt` and then run all the added goals in a loop so that `evaluate_goal`/`evaluate_all` is not necessary to call manually. There are a few complications for making everything work "right": 1. the `normalizes-to` hack that replaces the rhs with an unconstrained infer var requires special casing in the new `try_evaluate_added_goals` function similar to how `evaluate_goal`'s assertion special cases that hack. 2. `assemble_candidates_after_normalizing_self_ty`'s normalization step needs to be reran for each candidate otherwise the found candidates will potentially get a more accurate `QueryResult` when rerunning the projection/trait goal which can effect the `QueryResult` of the projection/trait goal. This is implemented via `EvalCtxt::probe`'s closure's `EvalCtxt` inheriting the added goals of the `EvalCtxt` that `probe` is called on, allowing us to add goals in a probe, and then enter a nested probe for each candidate and evaluate added goals which include the normalization step's goals. I made `make_canonical_response` evaluate added goals so that it will be hard to mess up the impl of the solver by forgetting to evaluate added goals. Right now the only way to mess this up would be to call `response_no_constraints` (which from the name is obviously weird). The visibility of `evaluate_goal` means that it can be called from various `compute_x_goal` or candidate assembly functions, this is generally wrong and we should never call `evaluate_goal` manually, instead we should be calling `add_goal`/`add_goals`. This is solved by moving `evaluate_goal` `evaluate_canonical_goal` and `compute_goal` into `eval_ctxt`'s module and making them private so they cannot be called from elsewhere, forcing people to call `add_goal/s` and `evaluate_added_goals_and_make_canonical_resposne`/`try_evaluate_added_goals` --- Other changes: - removed the `&& false` that was introduced to the assertion in `evaluate_goal` in rust-lang#108839 - remove a `!self.did_overflow()` requirement in `search_graph.is_empty()` which causes goals that overflow to ICE - made `EvalCtxt::eq` take `&mut self` and add all the nested goals via `add_goals` instead of returning them as 99% of call sites just immediately called `EvalCtxt::add_goals` manually. r? ```@lcnr```

…lcnr new solver: make all goal evaluation able to be automatically rerun It is generally wrong to call `evaluate_goal` multiple times or `evaluate_goal` and `evaluate_all` for the same `QueryResult` without correctly handling rerunning the goals when inference makes progress. Not doing so will result in the assertion in `evaluate_goal` firing because rerunning the goal will lead to a more accurate `QueryResult`. Currently there are lots of places that get this wrong and generally it is complex and error prone to handle correctly everywhere. This PR introduces a way to add goals to the `EvalCtxt` and then run all the added goals in a loop so that `evaluate_goal`/`evaluate_all` is not necessary to call manually. There are a few complications for making everything work "right": 1. the `normalizes-to` hack that replaces the rhs with an unconstrained infer var requires special casing in the new `try_evaluate_added_goals` function similar to how `evaluate_goal`'s assertion special cases that hack. 2. `assemble_candidates_after_normalizing_self_ty`'s normalization step needs to be reran for each candidate otherwise the found candidates will potentially get a more accurate `QueryResult` when rerunning the projection/trait goal which can effect the `QueryResult` of the projection/trait goal. This is implemented via `EvalCtxt::probe`'s closure's `EvalCtxt` inheriting the added goals of the `EvalCtxt` that `probe` is called on, allowing us to add goals in a probe, and then enter a nested probe for each candidate and evaluate added goals which include the normalization step's goals. I made `make_canonical_response` evaluate added goals so that it will be hard to mess up the impl of the solver by forgetting to evaluate added goals. Right now the only way to mess this up would be to call `response_no_constraints` (which from the name is obviously weird). The visibility of `evaluate_goal` means that it can be called from various `compute_x_goal` or candidate assembly functions, this is generally wrong and we should never call `evaluate_goal` manually, instead we should be calling `add_goal`/`add_goals`. This is solved by moving `evaluate_goal` `evaluate_canonical_goal` and `compute_goal` into `eval_ctxt`'s module and making them private so they cannot be called from elsewhere, forcing people to call `add_goal/s` and `evaluate_added_goals_and_make_canonical_resposne`/`try_evaluate_added_goals` --- Other changes: - removed the `&& false` that was introduced to the assertion in `evaluate_goal` in rust-lang#108839 - remove a `!self.did_overflow()` requirement in `search_graph.is_empty()` which causes goals that overflow to ICE - made `EvalCtxt::eq` take `&mut self` and add all the nested goals via `add_goals` instead of returning them as 99% of call sites just immediately called `EvalCtxt::add_goals` manually. r? ````@lcnr````

Rollup of 10 pull requests Successful merges: - rust-lang#106434 (Document `Iterator::sum/product` for Option/Result) - rust-lang#108326 (Implement read_buf for a few more types) - rust-lang#108842 (Enforce non-lifetime-binders in supertrait preds are not object safe) - rust-lang#108896 (new solver: make all goal evaluation able to be automatically rerun ) - rust-lang#109124 (Add `dist.compression-profile` option to control compression speed) - rust-lang#109240 (Walk un-shifted nested `impl Trait` in trait when setting up default trait method assumptions) - rust-lang#109385 (fix typo) - rust-lang#109386 (add myself to mailmap) - rust-lang#109390 (Custom MIR: Support aggregate expressions) - rust-lang#109408 (not *all* retags might be explicit in Runtime MIR) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

compiler/rustc_trait_selection/src/solve/project_goals.rs

rustbot assigned lcnr Mar 8, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver) labels Mar 8, 2023

BoxyUwU force-pushed the new_solver_add_goal_fn branch 3 times, most recently from c2ccd21 to c95a1a6 Compare March 9, 2023 15:43

BoxyUwU force-pushed the new_solver_add_goal_fn branch from d0c88bd to 00e91d4 Compare March 16, 2023 13:45

This comment has been minimized.

Sign in to view

replace usage of evaluate_goal with a new add_goal

ed63201

BoxyUwU force-pushed the new_solver_add_goal_fn branch from 1aa44d6 to ed63201 Compare March 16, 2023 14:58

lcnr reviewed Mar 17, 2023

View reviewed changes

BoxyUwU and others added 3 commits March 17, 2023 13:33

review nits

ee31e5f

add assert

ea08d3a

add comments

ce14a1e

Co-authored-by: lcnr <rust@lcnr.de>

BoxyUwU force-pushed the new_solver_add_goal_fn branch from e6f21f6 to ce14a1e Compare March 17, 2023 13:38

BoxyUwU added 2 commits March 17, 2023 14:04

UNACEPTABLE

e06c62c

fix let else unformatting

aa8de17

lcnr reviewed Mar 17, 2023

View reviewed changes

compiler/rustc_trait_selection/src/solve/mod.rs Outdated Show resolved Hide resolved

compiler/rustc_trait_selection/src/solve/trait_goals.rs Show resolved Hide resolved

BoxyUwU added 2 commits March 17, 2023 14:35

replace chain with two add_goal

e624ef4

fix bad assertion

9df35a5

BoxyUwU force-pushed the new_solver_add_goal_fn branch from 480ddf0 to 9df35a5 Compare March 17, 2023 14:40

move compute_goal and evaluate_x methods to inner module

b85bc19

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 20, 2023

Dylan-DPC mentioned this pull request Mar 21, 2023

Rollup of 10 pull requests #109428

Closed

matthiaskrgr mentioned this pull request Mar 21, 2023

Rollup of 10 pull requests #109430

Closed

matthiaskrgr mentioned this pull request Mar 21, 2023

Rollup of 9 pull requests #109431

Closed

Noratrieb mentioned this pull request Mar 21, 2023

Rollup of 14 pull requests #109439

Closed

Noratrieb mentioned this pull request Mar 21, 2023

Rollup of 10 pull requests #109442

Merged

bors merged commit e79b182 into rust-lang:master Mar 21, 2023

rustbot added this to the 1.70.0 milestone Mar 21, 2023

compiler-errors reviewed Mar 25, 2023

View reviewed changes

compiler/rustc_trait_selection/src/solve/project_goals.rs Show resolved Hide resolved

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

new solver: make all goal evaluation able to be automatically rerun #108896

new solver: make all goal evaluation able to be automatically rerun #108896

Uh oh!

BoxyUwU commented Mar 8, 2023 •

edited

Loading

Uh oh!

rustbot commented Mar 8, 2023

Uh oh!

bors commented Mar 11, 2023

Uh oh!

This comment has been minimized.

lcnr left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lcnr commented Mar 20, 2023

Uh oh!

bors commented Mar 20, 2023

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

new solver: make all goal evaluation able to be automatically rerun #108896

new solver: make all goal evaluation able to be automatically rerun #108896

Uh oh!

Conversation

BoxyUwU commented Mar 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Mar 8, 2023

Uh oh!

bors commented Mar 11, 2023

Uh oh!

This comment has been minimized.

lcnr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lcnr commented Mar 20, 2023

Uh oh!

bors commented Mar 20, 2023

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

BoxyUwU commented Mar 8, 2023 •

edited

Loading