Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Still more ObligationForest improvements. #64805

Merged
merged 11 commits into from
Oct 2, 2019
Merged

Still more ObligationForest improvements. #64805

merged 11 commits into from
Oct 2, 2019

Conversation

nnethercote
Copy link
Contributor

Following on from #64627, more readability improvements, but negligible effects on speed.

r? @nikomatsakis

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 26, 2019
@nnethercote
Copy link
Contributor Author

This is the end of my spate of ObligationForest changes. I didn't manage to get rid of the memcpys caused by the swap calls within compress, which was the original motivation for looking into ObligationForest. (Well... not without (a) regressing performance, (b) causing test failures, or (c) using unsafe.) drain_filter() is the right fit conceptually, but caused a slowdown in practice :(

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log. 
2019-09-26T07:20:08.1718424Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-26T07:20:08.1904525Z ##[command]git config gc.auto 0
2019-09-26T07:20:08.1998033Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-26T07:20:08.2060658Z ##[command]git config --get-all http.proxy
2019-09-26T07:20:08.2223453Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64805/merge:refs/remotes/pull/64805/merge
---
2019-09-26T08:27:04.2521920Z .................................................................................................... 1500/9044
2019-09-26T08:27:10.5786708Z .................................................................................................... 1600/9044
2019-09-26T08:27:23.4963227Z ..........................................................................i...............i......... 1700/9044
2019-09-26T08:27:30.6911019Z .................................................................................................... 1800/9044
2019-09-26T08:27:39.7858775Z .................................................................iiiii.............................. 1900/9044
2019-09-26T08:28:00.7517794Z .................................................................................................... 2100/9044
2019-09-26T08:28:03.4777073Z .................................................................................................... 2200/9044
2019-09-26T08:28:06.8425148Z .................................................................................................... 2300/9044
2019-09-26T08:28:15.9527139Z .................................................................................................... 2400/9044
---
2019-09-26T08:31:25.4262808Z ........................................................i...............i........................... 4700/9044
2019-09-26T08:31:35.2498035Z .................................................................................................... 4800/9044
2019-09-26T08:31:44.3020027Z .................................................................................................... 4900/9044
2019-09-26T08:31:52.2854541Z .................................................................................................... 5000/9044
2019-09-26T08:32:02.6221164Z ...........................................ii.ii.................................................... 5100/9044
2019-09-26T08:32:13.4139297Z .................................................................................................... 5300/9044
2019-09-26T08:32:24.4760863Z .................................................................................................... 5400/9044
2019-09-26T08:32:32.2826532Z ........i........................................................................................... 5500/9044
2019-09-26T08:32:38.2460043Z .................................................................................................... 5600/9044
2019-09-26T08:32:38.2460043Z .................................................................................................... 5600/9044
2019-09-26T08:32:50.6272060Z .................................................................................................... 5700/9044
2019-09-26T08:33:04.2621025Z ...ii...i..ii............i.......................................................................... 5800/9044
2019-09-26T08:33:27.3196761Z .................................................................................................... 6000/9044
2019-09-26T08:33:37.1608865Z .................................................................................................... 6100/9044
2019-09-26T08:33:37.1608865Z .................................................................................................... 6100/9044
2019-09-26T08:33:53.8213664Z .....i..ii.......................................................................................... 6200/9044
2019-09-26T08:34:14.1316332Z .................................................................i.................................. 6400/9044
2019-09-26T08:34:16.4682969Z .................................................................................................... 6500/9044
2019-09-26T08:34:19.2523721Z .....................................i.............................................................. 6600/9044
2019-09-26T08:34:23.5707037Z .................................................................................................... 6700/9044
---
2019-09-26T08:35:21.3521453Z .................................................................................................... 7200/9044
2019-09-26T08:35:27.1248564Z .................................................................................................... 7300/9044
2019-09-26T08:35:35.0538076Z .................................................................................................... 7400/9044
2019-09-26T08:35:46.0213214Z .................................................................................................... 7500/9044
2019-09-26T08:35:57.0818971Z .........................................................................................ii......i.. 7600/9044
2019-09-26T08:36:09.4301033Z .................................................................................................... 7800/9044
2019-09-26T08:36:27.7952234Z .................................................................................................... 7900/9044
2019-09-26T08:36:36.7016606Z .................................................................................................... 8000/9044
2019-09-26T08:36:47.4054420Z .................................................................................................... 8100/9044
---
2019-09-26T08:39:12.4658297Z  finished in 5.766
2019-09-26T08:39:12.4806446Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:12.6761976Z 
2019-09-26T08:39:12.6762234Z running 150 tests
2019-09-26T08:39:16.2542326Z i....iii......iii..iiii....i.............................i..i..................i....i.........ii.i.i 100/150
2019-09-26T08:39:18.3635693Z ..iiii..............i.........iii.i.......ii......
2019-09-26T08:39:18.3638395Z 
2019-09-26T08:39:18.3643131Z  finished in 5.883
2019-09-26T08:39:18.3829952Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:18.5572358Z 
---
2019-09-26T08:39:20.7414149Z  finished in 2.358
2019-09-26T08:39:20.7628910Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:20.9311079Z 
2019-09-26T08:39:20.9338195Z running 9 tests
2019-09-26T08:39:20.9339024Z iiiiiiiii
2019-09-26T08:39:20.9339402Z 
2019-09-26T08:39:20.9339482Z  finished in 0.168
2019-09-26T08:39:20.9533216Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:21.1459137Z 
---
2019-09-26T08:39:41.1289030Z  finished in 20.174
2019-09-26T08:39:41.1575421Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:41.3785766Z 
2019-09-26T08:39:41.3788665Z running 123 tests
2019-09-26T08:40:07.8578268Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....ii..........iiii..........i...ii...i.......ii. 100/123
2019-09-26T08:40:12.9879767Z i.i.i......iii.i.....ii
2019-09-26T08:40:12.9881676Z 
2019-09-26T08:40:12.9887896Z  finished in 31.834
2019-09-26T08:40:12.9901901Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:40:12.9902353Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-09-26T08:54:17.0429340Z 
2019-09-26T08:54:17.0430114Z    Doc-tests core
2019-09-26T08:54:22.5590872Z 
2019-09-26T08:54:22.5591270Z running 2405 tests
2019-09-26T08:54:34.6884481Z ......iiiii......................................................................................... 100/2405
2019-09-26T08:54:46.6125539Z ................................................................................ii.................. 200/2405
2019-09-26T08:55:14.5686089Z .i.................................................................................................. 400/2405
2019-09-26T08:55:14.5686089Z .i.................................................................................................. 400/2405
2019-09-26T08:55:26.1961600Z ................................................i..i.................iiii........................... 500/2405
2019-09-26T08:55:48.6866484Z .................................................................................................... 700/2405
2019-09-26T08:56:00.2353757Z .................................................................................................... 800/2405
2019-09-26T08:56:11.5510730Z .................................................................................................... 900/2405
2019-09-26T08:56:22.8585477Z .................................................................................................... 1000/2405
---
2019-09-26T09:00:48.0831131Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/mutex.rs:689:13
2019-09-26T09:00:48.0844677Z .......thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:00:48.0845128Z   left: `1`,
2019-09-26T09:00:48.0846002Z  right: `2`', src/libstd/sync/mutex.rs:653:13
2019-09-26T09:00:48.0913036Z ......thread '<unnamed>' panicked at 'test panic in inner thread to poison RwLock', src/libstd/sync/rwlock.rs:791:13
2019-09-26T09:00:48.0930478Z ...thread '.<unnamed>' panicked at 'test panic in inner thread to poison RwLock', src/libstd/sync/rwlock.rs:768:13
2019-09-26T09:00:48.0952645Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:705:13
2019-09-26T09:00:48.0961517Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:635:13
2019-09-26T09:00:48.0965441Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:646:13
2019-09-26T09:00:48.0972700Z ..thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:611:13
2019-09-26T09:00:50.1621647Z ..........................thread '<unnamed>' panicked at 'What the answer to my lifetimes dilemma is?', src/libstd/sys_common/remutex.rs:233:13
2019-09-26T09:00:50.1667530Z ........................ 700/763
2019-09-26T09:00:50.1772633Z ...................................thread '<unnamed>' panicked at 'explicit panic', src/libstd/thread/mod.rs:1535:13
2019-09-26T09:00:50.7859922Z ............thread '<unnamed>' panicked at 'Box<Any>', src/libstd/thread/mod.rs:1667:13
---
2019-09-26T09:00:57.5786006Z 
2019-09-26T09:00:57.5787324Z running 992 tests
2019-09-26T09:01:20.7614874Z i................................................................................................... 100/992
2019-09-26T09:01:33.5081082Z .................................................................................................... 200/992
2019-09-26T09:01:42.4907211Z .................iii......i......i...i......i....................................................... 300/992
2019-09-26T09:01:48.7715508Z .................................................................................................... 400/992
2019-09-26T09:01:57.3470795Z ...................................i..i.................................ii.......................... 500/992
2019-09-26T09:02:14.5783242Z .................................................................................................... 700/992
2019-09-26T09:02:14.5783242Z .................................................................................................... 700/992
2019-09-26T09:02:23.9195530Z ..................iiii.............................................................................. 800/992
2019-09-26T09:02:40.7113266Z .................................................................................................... 900/992
2019-09-26T09:02:49.2074246Z ........................................iiii................................................
2019-09-26T09:02:49.2075723Z 
2019-09-26T09:02:49.2169224Z  finished in 212.065
2019-09-26T09:02:49.2186343Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T09:02:49.4418675Z    Compiling term v0.0.0 (/checkout/src/libterm)
---
2019-09-26T09:05:03.4344108Z     Finished release [optimized] target(s) in 17.89s
2019-09-26T09:05:03.4385425Z      Running build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_data_structures-5f9a32e9b4b8b904
2019-09-26T09:05:03.4409830Z 
2019-09-26T09:05:03.4410192Z running 154 tests
2019-09-26T09:05:03.4776721Z .................................FFFF.F............................................................. 100/154
2019-09-26T09:05:03.4905546Z failures:
2019-09-26T09:05:03.4905698Z 
2019-09-26T09:05:03.4906507Z ---- obligation_forest::tests::diamond stdout ----
2019-09-26T09:05:03.4906507Z ---- obligation_forest::tests::diamond stdout ----
2019-09-26T09:05:03.4908483Z thread 'obligation_forest::tests::diamond' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4908859Z   left: `["A", "A.1", "A.2", "D"]`,
2019-09-26T09:05:03.4909384Z  right: `["D", "A.2", "A.1", "A"]`', src/librustc_data_structures/obligation_forest/tests.rs:264:5
2019-09-26T09:05:03.4909770Z 
2019-09-26T09:05:03.4910179Z ---- obligation_forest::tests::done_dependency stdout ----
2019-09-26T09:05:03.4910179Z ---- obligation_forest::tests::done_dependency stdout ----
2019-09-26T09:05:03.4910634Z thread 'obligation_forest::tests::done_dependency' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4911321Z   left: `["A: Sized", "B: Sized", "C: Sized"]`,
2019-09-26T09:05:03.4911906Z  right: `["C: Sized", "B: Sized", "A: Sized"]`', src/librustc_data_structures/obligation_forest/tests.rs:326:5
2019-09-26T09:05:03.4912479Z ---- obligation_forest::tests::orphan stdout ----
2019-09-26T09:05:03.4912479Z ---- obligation_forest::tests::orphan stdout ----
2019-09-26T09:05:03.4912941Z thread 'obligation_forest::tests::orphan' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4913133Z   left: `["C1", "C2"]`,
2019-09-26T09:05:03.4913530Z  right: `["C2", "C1"]`', src/librustc_data_structures/obligation_forest/tests.rs:364:5
2019-09-26T09:05:03.4913725Z 
2019-09-26T09:05:03.4914108Z ---- obligation_forest::tests::push_pop stdout ----
2019-09-26T09:05:03.4914544Z thread 'obligation_forest::tests::push_pop' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4914753Z   left: `["A.1", "A.3", "A.3.i"]`,
2019-09-26T09:05:03.4915187Z  right: `["A.3", "A.1", "A.3.i"]`', src/librustc_data_structures/obligation_forest/tests.rs:119:5
2019-09-26T09:05:03.4915757Z ---- obligation_forest::tests::success_in_grandchildren stdout ----
2019-09-26T09:05:03.4915757Z ---- obligation_forest::tests::success_in_grandchildren stdout ----
2019-09-26T09:05:03.4916221Z thread 'obligation_forest::tests::success_in_grandchildren' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4916432Z   left: `["A.1", "A.3"]`,
2019-09-26T09:05:03.4916833Z  right: `["A.3", "A.1"]`', src/librustc_data_structures/obligation_forest/tests.rs:175:5
2019-09-26T09:05:03.4917230Z 
2019-09-26T09:05:03.4917505Z failures:
2019-09-26T09:05:03.4917640Z     obligation_forest::tests::diamond
2019-09-26T09:05:03.4917789Z     obligation_forest::tests::done_dependency
2019-09-26T09:05:03.4917789Z     obligation_forest::tests::done_dependency
2019-09-26T09:05:03.4918304Z     obligation_forest::tests::orphan
2019-09-26T09:05:03.4918447Z     obligation_forest::tests::push_pop
2019-09-26T09:05:03.4918606Z     obligation_forest::tests::success_in_grandchildren
2019-09-26T09:05:03.4918728Z 
2019-09-26T09:05:03.4918870Z test result: FAILED. 149 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out
2019-09-26T09:05:03.4918992Z 
2019-09-26T09:05:03.4919454Z error: test failed, to rerun pass '-p rustc_data_structures --lib'
2019-09-26T09:05:03.4963972Z 
2019-09-26T09:05:03.4965023Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "-Zconfig-profile" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--locked" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "-p" "rustc_data_structures" "--" "--quiet"
2019-09-26T09:05:03.4966835Z expected success, got: exit code: 101
2019-09-26T09:05:03.4967760Z 
2019-09-26T09:05:03.4967760Z 
2019-09-26T09:05:03.4967936Z 
2019-09-26T09:05:03.4968094Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-09-26T09:05:03.4971688Z Build completed unsuccessfully in 1:37:14
2019-09-26T09:05:03.4972006Z == clock drift check ==
2019-09-26T09:05:03.4998863Z   local time: Thu Sep 26 09:05:03 UTC 2019
2019-09-26T09:05:03.6527725Z   network time: Thu, 26 Sep 2019 09:05:03 GMT
2019-09-26T09:05:03.6530378Z == end clock drift check ==
2019-09-26T09:05:04.3452982Z ##[error]Bash exited with code '1'.
2019-09-26T09:05:04.3503490Z ##[section]Starting: Checkout
2019-09-26T09:05:04.3505447Z ==============================================================================
2019-09-26T09:05:04.3505518Z Task         : Get sources
2019-09-26T09:05:04.3505566Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@pnkfelix
Copy link
Member

@nnethercote what sort of test failures were you observing from (b) above?

Are we talking about injecting unsoundness bugs into compiler? Or changes to diagnostic output? Or something else?

@nnethercote
Copy link
Contributor Author

Are we talking about injecting unsoundness bugs into compiler? Or changes to diagnostic output? Or something else?

They were all changes in error messages that arose from changing the order of pending obligations. Some of them looked pretty trivial (e.g. change the order of two errors) but some of them were more complex, to the point where I was struggling to tell if the new messages were as valid as the old ones. And there were quite a few in the latter basket. So I ended up preserving the order because the potential speed benefits of reordering were small.

@nnethercote
Copy link
Contributor Author

The test failures on automation are trivial; some vectors of erroneous obligations have changed order, but (AIUI) the order is not significant. I will fix those on Monday.

@nnethercote
Remove NodeState::OnDfsStack.
3b506d9 
It's not necessary; cycles (which are rare) can be detected by looking
at the node stack.

This change speeds things up slightly, as well as simplifying the code a
little.
@nnethercote
Rename node_index as index.
4a7fb8b 
For consistency with other variables in this file.
@nnethercote
Inline mark_as_waiting_from.
85a56cb 
It has a single call site, and the code is easier to read this way.
@nnethercote
Tweak the control flow in process_obligations().
7130e6c
@nnethercote
Use filter and map in to_errors.
ea72650
@nnethercote
Remove unnecessary uses of ObligationForest::scratch.
22943ee 
They don't help performance at all, and just complicate things.
@nnethercote
Introduce some intermediate variables that aid readability.
6fb1f37
@nnethercote
Convert some match expressions to ifs.
9e67f19 
These make the code more concise.
@nnethercote
Rename nodes_len and use it in a few more places.
8a62bb1
@nnethercote
Remove an out-of-date sentence in a comment.
2883c25
@nnethercote
Avoid the popping loop at the end of compress().
a820672 
By collecting the done obligations (when necessary) in the main loop.
This makes the code cleaner.

The commit also changes the order in which successful obligations are
returned -- they are now returned in the registered order, rather than
reversed. Because this order doesn't actually matter, being only used by
tests, the commit uses `sort()` to make the test agnostic w.r.t. the
order.
@nnethercote
Copy link
Contributor Author

I fixed the test failures and rebased. Ready for review, @nikomatsakis!

@nikomatsakis
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Oct 1, 2019

📌 Commit a820672 has been approved by nikomatsakis

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 1, 2019
tmandry added a commit to tmandry/rust that referenced this pull request Oct 2, 2019
@tmandry
Rollup merge of rust-lang#64805 - nnethercote:ObligForest-still-more,…
0e88e56 
… r=nikomatsakis

Still more `ObligationForest` improvements.

Following on from rust-lang#64627, more readability improvements, but negligible effects on speed.

r? @nikomatsakis
@tmandry tmandry mentioned this pull request Oct 2, 2019
Merged
bors added a commit that referenced this pull request Oct 2, 2019
@bors
Auto merge of #64981 - tmandry:rollup-slfkhay, r=tmandry
f2023ac 
Rollup of 11 pull requests

Successful merges:

 - #64649 (Avoid ICE on return outside of fn with literal array)
 - #64722 (Make all alt builders produce parallel-enabled compilers)
 - #64801 (Avoid `chain()` in `find_constraint_paths_between_regions()`.)
 - #64805 (Still more `ObligationForest` improvements.)
 - #64840 (SelfProfiler API refactoring and part one of event review)
 - #64885 (use try_fold instead of try_for_each to reduce compile time)
 - #64942 (Fix clippy warnings)
 - #64952 (Update cargo.)
 - #64974 (Fix zebra-striping in generic dataflow visualization)
 - #64978 (Fully clear `HandlerInner` in `Handler::reset_err_count`)
 - #64979 (Update books)

Failed merges:

 - #64959 (syntax: improve parameter without type suggestions)

r? @ghost
@bors bors merged commit a820672 into rust-lang:master Oct 2, 2019
@nnethercote nnethercote deleted the ObligForest-still-more branch April 2, 2020 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nikomatsakis nikomatsakis nikomatsakis approved these changes

Assignees

@nikomatsakis nikomatsakis

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@nnethercote @rust-highfive @pnkfelix @nikomatsakis @bors