Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements RFC 16, attributes on statements and expressions. #29850

Merged
merged 12 commits into from
Dec 4, 2015

Conversation

Kimundi
Copy link
Member

@Kimundi Kimundi commented Nov 15, 2015

See rust-lang/rfcs#16 and #15701

  • Added syntax support for attributes on expressions and all syntax nodes in statement position.
  • Extended #[cfg] folder to allow removal of statements, and
    of expressions in optional positions like expression lists and trailing
    block expressions.
  • Extended lint checker to recognize lint levels on expressions and
    locals.
  • As per RFC, attributes are not yet accepted on if expressions.

Examples:

#[cfg(unset)]
let x = y;
#[cfg(unset)]
{
        ...
}
assert_eq!((1, #[cfg(unset)] 2, 3), (1, 3));

#[allow(non_snake_case_name)]
let FOO = 0;

Implementation wise, there are a few rough corners and open questions:

  • The parser work ended up a bit ugly.
  • The pretty printer change was based mostly on guessing.
  • Similar to the if case, there are some places in the grammar where a new Expr node starts,
    but where it seemed weird to accept attributes and hence the parser doesn't. This includes:
    • const expressions in patterns
    • in the middle of an postfix operator chain (that is, after ., before indexing, before calls)
    • on range expressions, since #[attr] x .. y parses as (#[attr] x) .. y, which is inconsistent with
      #[attr] .. y which would parse as #[attr] (.. y)
  • Attributes are added as additional Option<Box<Vec<Attribute>>> fields in expressions and locals.
  • Memory impact has not been measured yet.
  • A cfg-away trailing expression in a block does not currently promote the previous StmtExpr in a block to a new trailing expr. That is to say, this won't work:
let x = {
    #[cfg(foo)]
    Foo { data: x }
    #[cfg(not(foo))]
    Foo { data: y }
};
  • One-element tuples can have their inner expression removed to become Unit, but just Parenthesis can't. Eg, (#[cfg(unset)] x,) == () but (#[cfg(unset)] x) == error. This seemed reasonable to me since tuples and unit are type constructors, but could probably be argued either way.
  • Attributes on macro nodes are currently unconditionally dropped during macro expansion, which seemed fine since macro disappear at that point?
  • Attributes on ast::ExprParens will be prepend-ed to the inner expression in the hir folder.
  • The work on pretty printer tests for this did trigger, but not fix errors regarding macros:
    • expression foo![] prints as foo!()
    • expression foo!{} prints as foo!()
    • statement foo![]; prints as foo!();
    • statement foo!{}; prints as foo!();
    • statement foo!{} triggers a None unwrap ICE.

@rust-highfive
Copy link
Collaborator

r? @pnkfelix

(rust_highfive has picked a reviewer for you, use r? to override)

@eddyb
Copy link
Member

eddyb commented Nov 15, 2015

cc @nikomatsakis
If libsyntax still has unit tests, could they be moved to src/test/run-pass-fulldeps/, where this PR adds a few tests invoking libsyntax?
It seems much cleaner than unit tests, and fixing them doesn't trigger a libsyntax rebuild (though that's just a minor annoyance that is solved by touch-ing a file into the past).

@nrc
Copy link
Member

nrc commented Nov 15, 2015

Is there a feature gate? It doesn't look like it from the tests, but it should have one.

@Kimundi
Copy link
Member Author

Kimundi commented Nov 16, 2015

Oh! Right, feature gate. Where do I add that?

@nrc
Copy link
Member

nrc commented Nov 16, 2015

@alexcrichton
Copy link
Member

It might be good to do a few measurements of this along the lines of:

  • Does this increase memory usage of compiling large programs like the compiler?
  • Does this slow down passes because of larger nodes? (I would suspect no)
  • An interesting project would be to attempt to put this into src/grammar before going too much farther (but it probably fits ok)

Thanks for the impl regardless @Kimundi!

@Kimundi
Copy link
Member Author

Kimundi commented Nov 16, 2015

Here's the result of compiling librustc with and without this patch:

rustc with stmt expr attrs:

time: 0.596; rss: 73MB  parsing
time: 0.176; rss: 73MB  configuration 1
time: 0.000; rss: 73MB  recursion limit
time: 0.017; rss: 73MB  gated macro checking
time: 0.000; rss: 73MB  crate injection
time: 0.028; rss: 75MB  macro loading
time: 0.000; rss: 75MB  plugin loading
time: 0.000; rss: 75MB  plugin registration
time: 2.619; rss: 112MB expansion
time: 0.079; rss: 112MB complete gated feature checking 1
time: 0.477; rss: 113MB configuration 2
time: 0.000; rss: 113MB gated configuration checking
time: 0.232; rss: 113MB maybe building test harness
time: 0.219; rss: 113MB prelude injection
time: 0.042; rss: 113MB checking that all macro invocations are gone
time: 0.000; rss: 113MB checking for inline asm in case the target doesn't support it
time: 0.078; rss: 113MB complete gated feature checking 2
time: 0.207; rss: 113MB assigning node ids
time: 0.208; rss: 178MB lowering ast -> hir
time: 0.139; rss: 192MB indexing hir
time: 0.000; rss: 192MB attribute checking
time: 0.109; rss: 192MB early lint checks
time: 0.042; rss: 224MB external crate/lib resolution
time: 0.059; rss: 224MB language item collection
time: 0.797; rss: 306MB resolution
time: 0.059; rss: 306MB lifetime resolution
time: 0.000; rss: 306MB looking for entry point
time: 0.031; rss: 306MB looking for plugin registrar
time: 0.213; rss: 328MB region resolution
time: 0.028; rss: 328MB loop checking
time: 0.030; rss: 328MB static item recursion checking
time: 0.263; rss: 336MB type collecting
time: 0.064; rss: 336MB variance inference
time: 0.331; rss: 357MB coherence checking
time: 0.364; rss: 362MB wf checking (old)
time: 0.678; rss: 364MB item-types checking
time: 20.812; rss: 514MB    item-bodies checking
time: 0.000; rss: 514MB drop-impl checking
time: 1.031; rss: 514MB wf checking (new)
time: 1.710; rss: 517MB const checking
time: 0.294; rss: 517MB privacy checking
time: 0.010; rss: 517MB stability index
time: 0.095; rss: 517MB intrinsic checking
time: 0.073; rss: 517MB effect checking
time: 0.317; rss: 517MB match checking
time: 1.838; rss: 646MB MIR dump
time: 0.166; rss: 650MB liveness checking
time: 2.271; rss: 650MB borrow checking
time: 1.840; rss: 650MB rvalue checking
time: 0.101; rss: 650MB reachability checking
time: 0.210; rss: 650MB death checking
time: 0.168; rss: 651MB stability checking
time: 0.000; rss: 651MB unused lib feature checking
time: 1.347; rss: 651MB lint checking
time: 0.006; rss: 651MB resolving dependency formats
time: 40.322; rss: 1413MB   translation
  time: 10.886; rss: 1194MB llvm function passes [0]
  time: 257.967; rss: 1180MB    llvm module passes [0]
  time: 69.784; rss: 1164MB codegen passes [0]
  time: 0.006; rss: 1164MB  codegen passes [0]
time: 344.635; rss: 1164MB  LLVM passes
  time: 0.869; rss: 1166MB  running linker
time: 3.216; rss: 1166MB    linking

rustc without stmt expr attrs:

time: 0.548; rss: 71MB  parsing
time: 0.210; rss: 71MB  configuration 1
time: 0.000; rss: 71MB  recursion limit
time: 0.021; rss: 71MB  gated macro checking
time: 0.000; rss: 71MB  crate injection
time: 0.031; rss: 74MB  macro loading
time: 0.000; rss: 74MB  plugin loading
time: 0.000; rss: 74MB  plugin registration
time: 2.268; rss: 109MB expansion
time: 0.096; rss: 109MB complete gated feature checking 1
time: 0.387; rss: 109MB configuration 2
time: 0.000; rss: 109MB gated configuration checking
time: 0.190; rss: 109MB maybe building test harness
time: 0.178; rss: 109MB prelude injection
time: 0.041; rss: 109MB checking that all macro invocations are gone
time: 0.000; rss: 109MB checking for inline asm in case the target doesn't support it
time: 0.077; rss: 109MB complete gated feature checking 2
time: 0.187; rss: 109MB assigning node ids
time: 0.186; rss: 171MB lowering ast -> hir
time: 0.114; rss: 185MB indexing hir
time: 0.000; rss: 185MB attribute checking
time: 0.112; rss: 185MB early lint checks
time: 0.040; rss: 218MB external crate/lib resolution
time: 0.053; rss: 218MB language item collection
time: 0.805; rss: 299MB resolution
time: 0.047; rss: 299MB lifetime resolution
time: 0.000; rss: 299MB looking for entry point
time: 0.026; rss: 299MB looking for plugin registrar
time: 0.212; rss: 321MB region resolution
time: 0.030; rss: 321MB loop checking
time: 0.028; rss: 321MB static item recursion checking
time: 0.269; rss: 329MB type collecting
time: 0.062; rss: 329MB variance inference
time: 0.330; rss: 350MB coherence checking
time: 0.413; rss: 355MB wf checking (old)
time: 0.639; rss: 358MB item-types checking
time: 20.803; rss: 507MB    item-bodies checking
time: 0.000; rss: 507MB drop-impl checking
time: 0.925; rss: 507MB wf checking (new)
time: 1.753; rss: 509MB const checking
time: 0.267; rss: 509MB privacy checking
time: 0.008; rss: 509MB stability index
time: 0.086; rss: 509MB intrinsic checking
time: 0.065; rss: 509MB effect checking
time: 0.271; rss: 509MB match checking
time: 1.709; rss: 638MB MIR dump
time: 0.158; rss: 642MB liveness checking
time: 2.339; rss: 642MB borrow checking
time: 1.892; rss: 643MB rvalue checking
time: 0.092; rss: 643MB reachability checking
time: 0.215; rss: 643MB death checking
time: 0.170; rss: 644MB stability checking
time: 0.000; rss: 644MB unused lib feature checking
time: 1.167; rss: 644MB lint checking
time: 0.004; rss: 644MB resolving dependency formats
time: 40.198; rss: 1388MB   translation
  time: 11.186; rss: 1177MB llvm function passes [0]
  time: 249.835; rss: 1163MB    llvm module passes [0]
  time: 70.738; rss: 1147MB codegen passes [0]
  time: 0.007; rss: 1147MB  codegen passes [0]
time: 337.899; rss: 1147MB  LLVM passes
  time: 0.945; rss: 1149MB  running linker
time: 2.737; rss: 1149MB    linking

chart

@durka
Copy link
Contributor

durka commented Nov 19, 2015

Question about macros: is #[allow(wormholes)] 1 + 1 matched by $e:expr, #[$attr:meta] $e:expr, both, or neither?

Also, the trailing expression thing seems like it might be nice to get working, because let foo = { #[cfg(bar)] 1 #[cfg(not(bar))] 2}; seems like it could be useful (eye-bleeding notwithstanding), and if this gets merged with the result of that being foo == () then it would be a breaking change to "fix" it later. Though I guess you could maybe simulate it with something like let foo = { (#[cfg(bar)] 1, #[cfg(not(bar))] 2, ).0 };.

@Kimundi
Copy link
Member Author

Kimundi commented Nov 19, 2015

Good question about the macro case! I'm guessing it counts as the latter if that macro syntax is legal today, I'd have to test it.

And yeah, the more I think about it the more I think the trailing case should just be made to work as expected. (Though its indeed simulatable today)

@bors
Copy link
Contributor

bors commented Nov 19, 2015

☔ The latest upstream changes (presumably #29903) made this pull request unmergeable. Please resolve the merge conflicts.

@Kimundi Kimundi force-pushed the attributes_that_make_a_statement branch from 88c1eca to 9131502 Compare November 24, 2015 18:14
@Kimundi
Copy link
Member Author

Kimundi commented Nov 24, 2015

Okay, I added the feature gate and rebased on upstream changes.

@durka I tested the macro case, and it will accept both, depending only on order of the macro clauses. Not sure if that is an issue regarding macro stability...

@bors
Copy link
Contributor

bors commented Nov 26, 2015

☔ The latest upstream changes (presumably #30015) made this pull request unmergeable. Please resolve the merge conflicts.

@durka
Copy link
Contributor

durka commented Nov 26, 2015

@Kimundi I dunno about future-proofing, but I think that is the best outcome in terms of macro writing. Because by switching the order of macro arms, you can choose whether you want to match "any expr, with or without attrs" or catch the attr (in order to implement custom attrs or something).

nodes in statement position.

Extended #[cfg] folder to allow removal of statements, and
of expressions in optional positions like expression lists and trailing
block expressions.

Extended lint checker to recognize lint levels on expressions and
locals.
@Kimundi Kimundi force-pushed the attributes_that_make_a_statement branch from e2df4bc to 49e9974 Compare November 26, 2015 20:48
// merge attributes into the inner expression.
return lower_expr(lctx, ex).map(|mut ex| {
ex.attrs.update(|attrs| {
attrs.prepend(e.attrs.clone())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, it might be good to have a test exercising the ordering enforced by this code.

(I did see a test with a generic #[attr], along the lines of #[attr] ( #[attr] expr ), but I didn't see anything that ensured that a prepend is occurring here rather than an append.)

Having said that, it isn't really that big a deal (especially since it may be non-trivial to actually make such a test... I guess one could do it by applying an #[allow(foo)] at the outer point followed by a #[deny(foo)] on the inner, and vice-versa, and making sure one gets the expected effect.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea about how to test that, added a test for it.

// in case of a cfg attr.
//
// NB: This is intentionally not part of the fold_expr() function
// in order for fold_opt_expr() to be able to avoid this check
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I first read this comment, I interpreted the phrase "to be able to avoid this check" as some sort of efficiency concern (which admittedly seemed a little strange in this context).

But now I think I understand: fold_opt_expr must skip the check, because the check does not apply to optional expression contexts. Do I read that right?

(I'm not sure I can think of a better phrasing off-hand. It could be just me that strongly associates the phrase "avoid a check" with "micro-optimization"... :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. Maybe it could be better phrased, but I'm not sure how either, hm...

@bors
Copy link
Contributor

bors commented Dec 4, 2015

📌 Commit d06f480 has been approved by pnkfelix

@bors
Copy link
Contributor

bors commented Dec 4, 2015

⌛ Testing commit d06f480 with merge 77ed39c...

bors added a commit that referenced this pull request Dec 4, 2015
…kfelix

See rust-lang/rfcs#16 and #15701

- Added syntax support for attributes on expressions and all syntax nodes in statement position.
- Extended `#[cfg]` folder to allow removal of statements, and
of expressions in optional positions like expression lists and trailing
block expressions.
- Extended lint checker to recognize lint levels on expressions and
locals.
- As per RFC, attributes are not yet accepted on `if` expressions.

Examples:
  ```rust
let x = y;
{
        ...
}
assert_eq!((1, #[cfg(unset)] 2, 3), (1, 3));

let FOO = 0;
```

Implementation wise, there are a few rough corners and open questions:
- The parser work ended up a bit ugly.
- The pretty printer change was based mostly on guessing.
- Similar to the `if` case, there are some places in the grammar where a new `Expr` node starts,
  but where it seemed weird to accept attributes and hence the parser doesn't. This includes:
  - const expressions in patterns
  - in the middle of an postfix operator chain (that is, after `.`, before indexing, before calls)
  - on range expressions, since `#[attr] x .. y` parses as  `(#[attr] x) .. y`, which is inconsistent with
    `#[attr] .. y` which would parse as `#[attr] (.. y)`
- Attributes are added as additional `Option<Box<Vec<Attribute>>>` fields in expressions and locals.
- Memory impact has not been measured yet.
- A cfg-away trailing expression in a block does not currently promote the previous `StmtExpr` in a block to a new trailing expr. That is to say, this won't work:
```rust
let x = {
    #[cfg(foo)]
    Foo { data: x }
    #[cfg(not(foo))]
    Foo { data: y }
};
```
- One-element tuples can have their inner expression removed to become Unit, but just Parenthesis can't. Eg, `(#[cfg(unset)] x,) == ()` but `(#[cfg(unset)] x) == error`. This seemed reasonable to me since tuples and unit are type constructors, but could probably be argued either way.
- Attributes on macro nodes are currently unconditionally dropped during macro expansion, which seemed fine since macro disappear at that point?
- Attributes on `ast::ExprParens` will be prepend-ed to the inner expression in the hir folder.
- The work on pretty printer tests for this did trigger, but not fix errors regarding macros:
  - expression `foo![]` prints as `foo!()`
  - expression `foo!{}` prints as `foo!()`
  - statement `foo![];` prints as `foo!();`
  - statement `foo!{};` prints as `foo!();`
  - statement `foo!{}` triggers a `None` unwrap ICE.
@petrochenkov
Copy link
Contributor

Holy shit, it broke 4 of my 5 pending PRs :D
I suppose it will break plenty of plugins generating pieces of AST or overriding folder methods, so marking this with [breaking-change] would be helpful.

@pnkfelix
Copy link
Member

pnkfelix commented Dec 5, 2015

@petrochenkov yeah we debated this at the compiler team mtg; niko and I weren't sure changing libsyntax like this should be categorized as "breaking" the way other changes are. I guess I still might have injected the breaking-change text, I don't know, we didn't come up with a firm rule

sgrif added a commit to sgrif/quasi that referenced this pull request Dec 6, 2015
Attrs are now allowed in statement and expression contexts. This will of
course break the build, as syntex_syntax will need to be updated as
well.

I also noticed that the test suite doesn't check nightly, only stable.
Perhaps we should update to run both?
sgrif added a commit to sgrif/aster that referenced this pull request Dec 6, 2015
This will of course break the build on stable, as syntex_syntax will
need to be updated as well. The suite now passes on nightly, however.
Though the pull request upstream which broke this adds support for
attributes in statement and expression contexts, I've opted not to
actually support it here yet, as we presumably cannot handle that on
stable for at least 2 more Rust versions.

Fixes serde-deprecated#56
@nrc
Copy link
Member

nrc commented Dec 6, 2015

FTR, I think breaking changes to libsyntax should be marked [breaking-change] (some but not all have been in the past). I do think this clearly passes the test for benefit outweighs the cost of breakage.

@tomjakubowski
Copy link
Contributor

This broke json_macros, although the fix was pretty simple. tomjakubowski/json_macros@7ada367

erickt added a commit to serde-deprecated/quasi that referenced this pull request Dec 8, 2015
bors added a commit that referenced this pull request Jan 27, 2016
…richton

`LateContext` already does this, looks like this was just forgotten in #29850.

Found while investigating #30326 (but doesn't fix it)
codemountains pushed a commit to codemountains/dotenvx that referenced this pull request Dec 20, 2023
Fixes the cascade of dependency breakages introduced in
rust-lang/rust#29850
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants