rustc: Move bytecode compression into codegen #45399
Conversation
|
r? @arielb1 (rust_highfive has picked a reviewer for you, use r? to override) |
|
I'm sort of perpetually unhappy with how no matter what I do the management of filenames in the backend is confusing, but hopefully it's a little less confusing than before? |
kennytm
added
the
S-waiting-on-author
There was a problem hiding this comment.
Great find! The new parallelization and caching should definitely be a win here.
The only downside I can see is that this will increase the size of the incr. cache. But hopefully we can get rid of bitcode compression entirely at some point by switching to MIR-only RLIBs.
r=me modulo nits and fixing the test case.
src/librustc_trans/back/write.rs
Outdated
| @@ -1157,17 +1183,20 @@ fn execute_work_item(cgcx: &CodegenContext, | |||
| err)); | |||
| } | |||
| } | |||
| *slot = Some(obj_out); | |||
There was a problem hiding this comment.
Couldn't you do this already in the match arms above?
Oh actually we're no longer storing uncompressed bytecode in the cache (as that's not needed for the linking stage), but default builds (those without fancy |
We'll probably have to start doing that for ThinLTO, right? |
|
Heh I've thought very little about how ThinLTO interacts with incremental compilation :) In theory there's a lot of possible wins to gain there (fully incremental release builds) but I think we've got a good bit of support on the LLVM side of things to sort out before that works. |
alexcrichton
force-pushed
the
compress-parallel
branch
from
5a183e4 to
3bfa312
Compare
|
@bors: r=michaelwoerister |
|
📌 Commit 3bfa312 has been approved by |
kennytm
added
S-waiting-on-bors
alexcrichton
force-pushed
the
compress-parallel
branch
2 times, most recently
from
2effe64 to
9963d13
Compare
|
@bors: r=michaelwoerister |
|
📌 Commit 9963d13 has been approved by |
|
☔ The latest upstream changes (presumably #45348) made this pull request unmergeable. Please resolve the merge conflicts. |
kennytm
added
S-waiting-on-author
This commit moves compression of the bytecode from the `link` module to the `write` module, namely allowing it to be (a) cached by incremental compilation and (b) produced in parallel. The parallelization may show up as some nice wins during normal compilation and the caching in incremental mode should be beneficial for incremental compiles! (no more need to recompress the entire crate's bitcode on all builds)
alexcrichton
force-pushed
the
compress-parallel
branch
from
9963d13 to
8197a0b
Compare
|
@bors: r=michaelwoerister |
|
📌 Commit 8197a0b has been approved by |
…ster rustc: Move bytecode compression into codegen This commit moves compression of the bytecode from the `link` module to the `write` module, namely allowing it to be (a) cached by incremental compilation and (b) produced in parallel. The parallelization may show up as some nice wins during normal compilation and the caching in incremental mode should be beneficial for incremental compiles! (no more need to recompress the entire crate's bitcode on all builds)
|
☀️ Test successful - status-appveyor, status-travis |
|
This seems to have brought some nice performance improvements: http://perf.rust-lang.org/compare.html?start=4279e2b4c14dc90191595c97ad3a019d618e7158&end=9ea3878dfec9b20993eaebfdb7566b29e49c52ab&stat=wall-time But they were eaten up immediately by switching to ThinLTO in the next PR 😁😭 |
This commit moves compression of the bytecode from the
linkmodule to thewritemodule, namely allowing it to be (a) cached by incremental compilationand (b) produced in parallel. The parallelization may show up as some nice wins
during normal compilation and the caching in incremental mode should be
beneficial for incremental compiles! (no more need to recompress the entire
crate's bitcode on all builds)