Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc_mir: Hide initial block state when defining transfer functions #61787

Merged
merged 2 commits into from
Jun 24, 2019

Conversation

ecstatic-morse
Copy link
Contributor

@ecstatic-morse ecstatic-morse commented Jun 12, 2019

This PR addresses this FIXME.

This makes sets.on_entry inaccessible in {before_,}{statement,terminator}_effect. This field was meant to allow implementors of BitDenotation to access the initial state for each block (optionally with the effect of all previous statements applied via accumulates_intrablock_state) while defining transfer functions. However, the ability to set the initial value for the entry set of each basic block (except for START_BLOCK) no longer exists. As a result, this functionality is mostly useless, and when it was used it was used erroneously (see #62007).

Since on_entry is now useless, we can also remove BlockSets, which held the gen, kill, and on_entry bitvectors and replace it with a GenKill struct. Variables of this type are called trans since they represent a transfer function. GenKills are stored contiguously in AllSets, which reduces the number of bounds checks and may improve cache performance: one is almost never accessed without the other.

Replacing BlockSets with GenKill allows us to define some new helper functions which streamline dataflow iteration and the dataflow-at-location APIs. Notably, state_for_location used a subtle side-effect of the kill/kill_all setters to apply the transfer function, and could be incorrect if a transfer function depended on effects of previous statements in the block on gen_set.

Additionally, this PR merges BitSetOperator and InitialFlow into one trait. Since the value of InitialFlow defines the semantics of the join operation, there's no reason to have seperate traits for each. We can add a default impl of join which branches based on BOTTOM_VALUE. This should get optimized away.

@rust-highfive
Copy link
Collaborator

r? @petrochenkov

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 12, 2019
@Centril
Copy link
Contributor

Centril commented Jun 12, 2019

r? @eddyb cc @oli-obk

@rust-highfive rust-highfive assigned eddyb and unassigned petrochenkov Jun 12, 2019
src/librustc_mir/dataflow/impls/borrowed_locals.rs Outdated Show resolved Hide resolved
src/librustc_mir/dataflow/impls/mod.rs Outdated Show resolved Hide resolved
src/librustc_mir/dataflow/impls/mod.rs Outdated Show resolved Hide resolved
src/librustc_mir/dataflow/impls/mod.rs Outdated Show resolved Hide resolved
@bors
Copy link
Contributor

bors commented Jun 14, 2019

☔ The latest upstream changes (presumably #61817) made this pull request unmergeable. Please resolve the merge conflicts.

@eddyb
Copy link
Member

eddyb commented Jun 16, 2019

Does the "const-correctness" in the title refer to the C/++ concept, i.e. immutability?
I don't know how to rephrase it better, but either way I don't think this is related to @RalfJung's concepts around compile-time evaluation, @Centril.

r? @pnkfelix

@rust-highfive rust-highfive assigned pnkfelix and unassigned eddyb Jun 16, 2019
@Centril
Copy link
Contributor

Centril commented Jun 16, 2019

Does the "const-correctness" in the title refer to the C/++ concept, i.e. immutability?
I don't know how to rephrase it better, but either way I don't think this is related to @RalfJung's concepts around compile-time evaluation, @Centril.

I also don't know what to call things but "const-correctness" does seem confusing from a C/C++ POV if it isn't that?

@eddyb
Copy link
Member

eddyb commented Jun 16, 2019

To be clear: I don't think we should be using "const-correctness" to talk about mutability in Rust code.
But I don't have a suggestion for rephrasing the title of this PR.

@RalfJung
Copy link
Member

"refactor for more precise mutability information"?

@ecstatic-morse
Copy link
Contributor Author

Yes, I meant "const-correctness" in the C++ sense.

"Ensure entry_set is immutable while transfer functions are being defined"?

@ecstatic-morse ecstatic-morse changed the title librustc_mir: refactor dataflow for const-correctness librustc_mir: Make entry_set immutable when defining transfer functions Jun 16, 2019
@ecstatic-morse
Copy link
Contributor Author

I changed the title and made the names of the fields ofGenKill less terse.

@bors
Copy link
Contributor

bors commented Jun 18, 2019

☔ The latest upstream changes (presumably #61891) made this pull request unmergeable. Please resolve the merge conflicts.

@pnkfelix
Copy link
Member

@ecstatic-morse : Lets rebase this on top of PR ##62010 and then revise it to change the dataflow API to not pass entry-set at all.

Does that sound okay to you?

@pnkfelix pnkfelix added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 21, 2019
@ecstatic-morse
Copy link
Contributor Author

ecstatic-morse commented Jun 21, 2019

@pnkfelix Yep. Thanks for all the help on #62010 by the way! This means that accumulates_intrablock_state can go away as well?

@ecstatic-morse
Copy link
Contributor Author

@pnkfelix. This PR now removes accumulates_intrablock_state as well as entry_set. I was also able to refactor InitialFlow since it should be implemented by all dataflow analyses (I believe there was a time when one could explicitly initialize the entry set of each block, but this is no longer possible). Now it's much harder to define an incorrect join operation for the analysis.

See the summary at the top for a full description.

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:286f7d30:start=1561146146161932745,finish=1561146146990675399,duration=828742654
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---

[00:04:54] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:54] tidy error: /checkout/src/librustc_mir/dataflow/mod.rs:558: line longer than 100 chars
[00:04:59] some tidy checks failed
[00:04:59] 
[00:04:59] 
[00:04:59] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:59] 
[00:04:59] 
[00:04:59] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:59] Build completed unsuccessfully in 0:01:17
---
travis_time:end:16b669c0:start=1561146457883433897,finish=1561146457888369543,duration=4935646
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0eb1c162
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:038b2250
travis_time:start:038b2250
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:106c8300
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@ecstatic-morse ecstatic-morse changed the title librustc_mir: Make entry_set immutable when defining transfer functions rustc_mir: Hide initial block state when defining transfer functions Jun 21, 2019
This commit makes `sets.on_entry` inaccessible in
`{before_,}{statement,terminator}_effect`. This field was meant to allow
implementors of `BitDenotation` to access the initial state for each
block (optionally with the effect of all previous statements applied via
`accumulates_intrablock_state`) while defining transfer functions.
However, the ability to set the initial value for the entry set of each
basic block (except for START_BLOCK) no longer exists. As a result, this
functionality is mostly useless, and when it *was* used it was used
erroneously (see rust-lang#62007).

Since `on_entry` is now useless, we can also remove `BlockSets`, which
held the `gen`, `kill`, and `on_entry` bitvectors and replace it with a
`GenKill` struct. Variables of this type are called `trans` since they
represent a transfer function. `GenKill`s are stored contiguously in
`AllSets`, which reduces the number of bounds checks and may improve
cache performance: one is almost never accessed without the other.

Replacing `BlockSets` with `GenKill` allows us to define some new helper
functions which streamline dataflow iteration and the
dataflow-at-location APIs. Notably, `state_for_location` used a subtle
side-effect of the `kill`/`kill_all` setters to apply the transfer
function, and could be incorrect if a transfer function depended on
effects of previous statements in the block on `gen_set`.
Since the value of `InitialFlow` defines the semantics of the `join`
operation, there's no reason to have seperate traits for each. We can
add a default impl of `join` which branches based on `BOTTOM_VALUE`.
This should get optimized away.
@ecstatic-morse
Copy link
Contributor Author

@pnkfelix This is now rebased and ready for another round of review.

@pnkfelix pnkfelix added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jun 24, 2019
fn bottom_value() -> bool;
///
/// `BottomValue` determines whether the initial entry set for each basic block is empty or full.
/// This also determines the semantics of the lattice `join` operator used to merge dataflow
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, clever

@pnkfelix
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Jun 24, 2019

📌 Commit c8cbd4f has been approved by pnkfelix

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 24, 2019
@bors
Copy link
Contributor

bors commented Jun 24, 2019

⌛ Testing commit c8cbd4f with merge c873491e73405ff647abd78f06db6e699bafcdb0...

@bors
Copy link
Contributor

bors commented Jun 24, 2019

💔 Test failed - checks-travis

@rust-highfive
Copy link
Collaborator

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
Need to get 2,649 kB of archives.
After this operation, 7,904 kB of additional disk space will be used.
Ign:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace1 amd64 1.3.2-1
Ign:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace-ctf1 amd64 1.3.2-1
Err:3 http://security.ubuntu.com/ubuntu xenial-security/main amd64 gdb amd64 7.11.1-0ubuntu1~16.5
  Unable to connect to apt.cache.travis-ci.com:http:
Ign:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace-ctf1 amd64 1.3.2-1
Ign:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace1 amd64 1.3.2-1
Ign:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace-ctf1 amd64 1.3.2-1
Ign:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace1 amd64 1.3.2-1
Ign:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace1 amd64 1.3.2-1
Ign:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace-ctf1 amd64 1.3.2-1
Ign:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace1 amd64 1.3.2-1
Ign:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace-ctf1 amd64 1.3.2-1
Err:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace1 amd64 1.3.2-1
  Could not connect to apt.cache.travis-ci.com:80 (34.96.81.152), connection timed out
Err:2 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 libbabeltrace-ctf1 amd64 1.3.2-1
  Unable to connect to apt.cache.travis-ci.com:http:
Fetched 2,526 kB in 30s (81.5 kB/s)
Fetched 2,526 kB in 30s (81.5 kB/s)
E: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/pool/main/b/babeltrace/libbabeltrace1_1.3.2-1_amd64.deb  Could not connect to apt.cache.travis-ci.com:80 (34.96.81.152), connection timed out
E: Failed to fetch http://us-east-1.ec2.archive.ubuntu.com/ubuntu/pool/main/b/babeltrace/libbabeltrace-ctf1_1.3.2-1_amd64.deb  Unable to connect to apt.cache.travis-ci.com:http:
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
travis_fold:start:apt-get.diagnostics
apt-get install failed
apt-get install failed
$ cat ${TRAVIS_HOME}/apt-get-update.log
Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://apt.postgresql.org/pub/repos/apt xenial-pgdg InRelease [51.5 kB]
Get:4 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
Get:5 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:5 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:6 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 Packages [205 kB]
Get:7 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main i386 Packages [205 kB]
Get:8 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main Sources [1,103 kB]
Get:9 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/restricted Sources [5,179 B]
Get:10 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe Sources [9,802 kB]
Get:12 http://security.ubuntu.com/ubuntu xenial-security/main Sources [185 kB]
Get:13 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1,558 kB]
Get:14 http://security.ubuntu.com/ubuntu xenial-security/restricted Sources [2,243 B]
Get:15 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [131 kB]
Get:15 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [131 kB]
Get:16 http://security.ubuntu.com/ubuntu xenial-security/multiverse Sources [3,517 B]
Get:17 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [868 kB]
Get:18 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main i386 Packages [1,552 kB]
Get:19 http://security.ubuntu.com/ubuntu xenial-security/main i386 Packages [709 kB]
Get:20 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/main Translation-en [799 kB]
Get:21 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [14.1 kB]
Get:22 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/restricted i386 Packages [14.5 kB]
Get:23 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/restricted Translation-en [3,019 B]
Get:24 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [385 kB]
Get:25 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [9,827 kB]
Get:27 http://security.ubuntu.com/ubuntu xenial-security/restricted i386 Packages [12.7 kB]
Get:27 http://security.ubuntu.com/ubuntu xenial-security/restricted i386 Packages [12.7 kB]
Get:28 http://security.ubuntu.com/ubuntu xenial-security/restricted Translation-en [2,204 B]
Get:30 http://security.ubuntu.com/ubuntu xenial-security/universe i386 Packages [484 kB]
Get:31 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [242 kB]
Get:32 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [6,121 B]
Get:33 http://security.ubuntu.com/ubuntu xenial-security/multiverse i386 Packages [6,297 B]
Get:33 http://security.ubuntu.com/ubuntu xenial-security/multiverse i386 Packages [6,297 B]
Get:34 http://security.ubuntu.com/ubuntu xenial-security/multiverse Translation-en [2,699 B]
Get:35 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe i386 Packages [9,804 kB]
Get:36 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/universe Translation-en [6,256 kB]
Get:38 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse i386 Packages [172 kB]
Get:39 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial/multiverse Translation-en [131 kB]
Get:40 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Sources [425 kB]
Get:41 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted Sources [2,696 B]
Get:41 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted Sources [2,696 B]
Get:42 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe Sources [323 kB]
Get:43 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [9,428 B]
Get:44 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [1,261 kB]
Get:45 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main i386 Packages [1,076 kB]
Get:46 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [549 kB]
Get:47 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [13.1 kB]
Get:48 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted i386 Packages [13.1 kB]
Get:49 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/restricted Translation-en [2,337 B]
Get:51 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-updates/universe i386 Packages [887 kB]
Get:52 http://us-east-1.ec2.archive.ubuntu.com/ubuntu xenial-

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 24, 2019
@tesuji
Copy link
Contributor

tesuji commented Jun 24, 2019

Spurious failure.

@Centril
Copy link
Contributor

Centril commented Jun 24, 2019

@bors retry spurious network

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 24, 2019
@bors
Copy link
Contributor

bors commented Jun 24, 2019

⌛ Testing commit c8cbd4f with merge 8aa42ed...

bors added a commit that referenced this pull request Jun 24, 2019
…kfelix

rustc_mir: Hide initial block state when defining transfer functions

This PR addresses [this FIXME](https://github.com/rust-lang/rust/blob/2887008e0ce0824be4e0e9562c22ea397b165c97/src/librustc_mir/dataflow/mod.rs#L594-L596).

This makes `sets.on_entry` inaccessible in `{before_,}{statement,terminator}_effect`. This field was meant to allow implementors of `BitDenotation` to access the initial state for each block (optionally with the effect of all previous statements applied via `accumulates_intrablock_state`) while defining transfer functions.  However, the ability to set the initial value for the entry set of each basic block (except for START_BLOCK) no longer exists. As a result, this functionality is mostly useless, and when it *was* used it was used erroneously (see #62007).

Since `on_entry` is now useless, we can also remove `BlockSets`, which held the `gen`, `kill`, and `on_entry` bitvectors and replace it with a `GenKill` struct. Variables of this type are called `trans` since they represent a transfer function. `GenKill`s are stored contiguously in `AllSets`, which reduces the number of bounds checks and may improve cache performance: one is almost never accessed without the other.

Replacing `BlockSets` with `GenKill` allows us to define some new helper functions which streamline dataflow iteration and the dataflow-at-location APIs. Notably, `state_for_location` used a subtle side-effect of the `kill`/`kill_all` setters to apply the transfer function, and could be incorrect if a transfer function depended on effects of previous statements in the block on `gen_set`.

Additionally, this PR merges `BitSetOperator` and `InitialFlow` into one trait. Since the value of `InitialFlow` defines the semantics of the `join` operation, there's no reason to have seperate traits for each. We can add a default impl of `join` which branches based on `BOTTOM_VALUE`.  This should get optimized away.
@bors
Copy link
Contributor

bors commented Jun 24, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: pnkfelix
Pushing 8aa42ed to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jun 24, 2019
@bors bors merged commit c8cbd4f into rust-lang:master Jun 24, 2019
@ecstatic-morse ecstatic-morse deleted the dataflow-split-block-sets branch June 25, 2019 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants