Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the requirement that ast->hir lowering be reproducible #33296

Merged
merged 7 commits into from
May 2, 2016

Conversation

jseyfried
Copy link
Contributor

@jseyfried jseyfried commented Apr 30, 2016

This PR changes the ast->hir lowerer to be non-reproducible, and it removes the lowering context's id cache.

If the hir of an ast node needs to be reproduced, we can use the hir map instead of the lowerer -- for example, tcx.map.expect_expr(expr.id) instead of lower_expr(lcx, expr).

r? @nrc

@jseyfried jseyfried force-pushed the non_idempotent_lowering branch 2 times, most recently from b1b28c5 to ce9e894 Compare April 30, 2016 20:13
@nagisa
Copy link
Member

nagisa commented Apr 30, 2016

What’s the purpose of the change? Seems like a potentially plugin-breaking-change to me.

@petrochenkov
Copy link
Contributor

Side note: if librustc_save_analysis doesn't use AST -> HIR lowering anymore, then MTWT tables can be cleared even if save_analysis is enabled. See #30145 (comment)

@jseyfried
Copy link
Contributor Author

jseyfried commented Apr 30, 2016

@nagisa

What’s the purpose of the change?

The purpose is to simplify lowering (by removing the id caching system) and to remove the lowering context from the CompileState.

Also, it will be significantly simpler to do name resolution during lowering if lowering doesn't need to be reproducible. I don't think it's worth the extra complexity needed to be reproducible when we can just use the HIR map to re-lower (almost) any AST node.

@jseyfried
Copy link
Contributor Author

jseyfried commented Apr 30, 2016

@petrochenkov good point, I added a commit that clears the MTWT tables after lowering for save-analysis.

@jseyfried jseyfried changed the title Remove the requirement that ast->hir lowering be idempotent Remove the requirement that ast->hir lowering be reproducible May 1, 2016
@@ -2121,21 +2121,8 @@ fn signal_block_expr(lctx: &LoweringContext,

#[cfg(test)]
mod test {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can remove this whole module

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// This whole system relies on node ids being incremented one at a time and
// all increments being for lowering. This means that you should not call any
// non-lowering function which will use new node ids.
//
// We must also cache gensym'ed Idents to ensure that we get the same Ident
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment can probably go too

Copy link
Contributor Author

@jseyfried jseyfried May 2, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, forgot about that -- done.

@nrc
Copy link
Member

nrc commented May 2, 2016

r+ with that comment removed

@nrc
Copy link
Member

nrc commented May 2, 2016

@bors: r+

@bors
Copy link
Contributor

bors commented May 2, 2016

📌 Commit ef69ef8 has been approved by nrc

@jseyfried
Copy link
Contributor Author

@nrc Thanks!
I'm almost done with a PR for resolving AST, pending #33190.

@bors
Copy link
Contributor

bors commented May 2, 2016

⌛ Testing commit ef69ef8 with merge 855fb61...

bors added a commit that referenced this pull request May 2, 2016
Remove the requirement that ast->hir lowering be reproducible

This PR changes the ast->hir lowerer to be non-reproducible, and it removes the lowering context's id cache.

If the `hir` of an `ast` node needs to be reproduced, we can use the hir map instead of the lowerer -- for example, `tcx.map.expect_expr(expr.id)` instead of `lower_expr(lcx, expr)`.

r? @nrc
@bors bors merged commit ef69ef8 into rust-lang:master May 2, 2016
@RalfJung
Copy link
Member

RalfJung commented May 6, 2016

What exactly is getting non-reproducible with this change? Does this mean that if I compile the same Rust program twice, I would get two different binaries (as in, binaries that are not bit-for-bit identical)? That would be a huge bummer -- reproducible builds are a cornerstone of building trustworthy binaries. If Rust can't build reproducibly, This is actively used, for example, by the Tor project and some BitCoin clients. it would be unsuited for any application that needs to establish trust in the generated binaries. See https://reproducible-builds.org/ for more details.

@jseyfried
Copy link
Contributor Author

jseyfried commented May 6, 2016

@RalfJung

Does this mean that if I compile the same Rust program twice, I would get two different binaries (as in, binaries that are not bit-for-bit identical)?

I don't know if Rust has this property, but this PR isn't related to reproducible builds. Lowering is still deterministic after this PR (i.e. it behaves the same when building the same code).

This PR changed lowering so that if the same AST node is lowered multiple times during the same build, we aren't guaranteed to get same result. Perhaps a better word would be "idempotent" instead of "reproducible".

@jseyfried jseyfried deleted the non_idempotent_lowering branch May 7, 2016 06:34
Manishearth added a commit to Manishearth/rust that referenced this pull request Jun 29, 2016
…nique, r=nrc

Give `ast::ExprKind::Paren` no-op expressions the same ids as their children.

Having `ast::ExprKind::Paren` expressions share ids with their children
 - reduces the number of unused `NodeId`s in the hir map and
 - guarantees that `tcx.map.expect_expr(ast_expr.id)` is the hir corresponding to `ast_expr`.

This fixes the bug from rust-lang#34327, which was introduced in rust-lang#33296 when I assumed the above guarantee.

r? @nrc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants