Assign node ids during macro expansion #36438

jseyfried · 2016-09-13T03:45:58Z

After this PR,

The ExtCtxt can access resolve's Resolver through the trait object ext::base::Resolver.
- The Resolver trait object can load macros and replaces today's MacroLoader trait object.
- The macro expander uses the Resolver trait object to resolve macro invocations.
The macro expander assigns node ids and builds the Resolver's macros_at_scope map.
- This is groundwork for merging import resolution and expansion (cc Tracking issue for "macro naming and modularisation" (RFC #1561) #35896).
Performance of expansion together with node id assignment improves by ~5%.

EDIT: Since Github is reordering the commits, here is git log:

b54e1e3: Differentiate between monotonic and non-monotonic expansion and only assign node ids during monotonic expansion.
78c0039: Expand generated test harnesses and macro registries.
f3c2dca: Remove scope placeholders from the crate root.
c86c8d4: Perform node id assignment and macros_at_scope construction during the InvocationCollector and PlaceholderExpander folds.
72a6369: Move macro resolution into librustc_resolve.
20b43b2: Rewrite the unit tests in ext/expand.rs as a compile-fail test.
a9821e1: Refactor ExtCtxt to use a Resolver instead of a MacroLoader.
60440b2: Refactor noop_fold_stmt_kind out of noop_fold_stmt.
50f94f6: Avoid needless reexpansions.

jseyfried · 2016-09-13T03:53:54Z

src/libsyntax/ext/expand.rs

-
-    // create a really evil test case where a $x appears inside a binding of $x
-    // but *shouldn't* bind because it was inserted by a different macro....
-    // can't write this test case until we have macro-generating macros.


We now have a test like this -- c.f. #35453.

jseyfried · 2016-09-13T05:17:45Z

cc @eddyb

eddyb · 2016-09-13T05:29:30Z

src/librustc/middle/cstore.rs

@@ -488,6 +489,9 @@ impl<'tcx> CrateStore<'tcx> for DummyCrateStore {
    fn metadata_encoding_version(&self) -> &[u8] { bug!("metadata_encoding_version") }
 }

+pub trait MacroLoader {
+     fn load_crate(&mut self, _: &ast::Item, _: bool) -> Vec<LoadedMacro>;


May be useful to specify argument names.

Good point, done.

the `InvocationCollector` and `PlaceholderExpander` folds.

only assign node ids during monotonic expansion.

nrc · 2016-09-14T04:35:54Z

@bors: r+

bors · 2016-09-14T04:35:56Z

📌 Commit b54e1e3 has been approved by nrc

bors · 2016-09-14T04:35:58Z

⌛ Testing commit b54e1e3 with merge bcf6bd2...

bors · 2016-09-14T04:42:02Z

💔 Test failed - auto-win-gnu-32-opt-rustbuild

alexcrichton · 2016-09-14T05:42:35Z

@bors: retry

On Tue, Sep 13, 2016 at 9:42 PM, bors notifications@github.com wrote:

💔 Test failed - auto-win-gnu-32-opt-rustbuild
https://buildbot.rust-lang.org/builders/auto-win-gnu-32-opt-rustbuild/builds/2504

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#36438 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAD95Ai7ZBb7A_lcI1k0I3kn7aSEJIUmks5qp3scgaJpZM4J7Q2c
.

@nrc

Assign node ids during macro expansion After this PR, - The `ExtCtxt` can access `resolve`'s `Resolver` through the trait object `ext::base::Resolver`. - The `Resolver` trait object can load macros and replaces today's `MacroLoader` trait object. - The macro expander uses the `Resolver` trait object to resolve macro invocations. - The macro expander assigns node ids and builds the `Resolver`'s `macros_at_scope` map. - This is groundwork for merging import resolution and expansion. - Performance of expansion together with node id assignment improves by ~5%. **EDIT:** Since Github is reordering the commits, here is `git log`: - b54e1e3: Differentiate between monotonic and non-monotonic expansion and only assign node ids during monotonic expansion. - 78c0039: Expand generated test harnesses and macro registries. - f3c2dca: Remove scope placeholders from the crate root. - c86c8d4: Perform node id assignment and `macros_at_scope` construction during the `InvocationCollector` and `PlaceholderExpander` folds. - 72a6369: Move macro resolution into `librustc_resolve`. - 20b43b2: Rewrite the unit tests in `ext/expand.rs` as a `compile-fail` test. - a9821e1: Refactor `ExtCtxt` to use a `Resolver` instead of a `MacroLoader`. - 60440b2: Refactor `noop_fold_stmt_kind` out of `noop_fold_stmt`. - 50f94f6: Avoid needless reexpansions. r? @nrc

Rollup of 9 pull requests - Successful merges: #36384, #36405, #36425, #36429, #36438, #36454, #36459, #36461, #36463 - Failed merges: #36444

Originally, there has been a dedicated pass for renumbering AST NodeIds to have actual values. This pass had been added by commit a5ad4c3. Then, later, this step was moved to where it resides now, macro expansion. See commit c86c8d4 or PR rust-lang#36438. The comment snippet, added by the original commit, has survived the times without any change, becoming outdated at removal of the dedicated pass. Nowadays, grepping for the next_node_id function will show up multiple places in the compiler that call it, but the main rewriting that the comment talks about is still done in the expansion step, inside an innocious looking visit_id function that's called during macro invocation collection.

…=oli-obk The renumber pass is long gone Originally, there has been a dedicated pass for renumbering AST NodeIds to have actual values. This pass had been added by commit a5ad4c3. Then, later, this step was moved to where it resides now, macro expansion. See commit c86c8d4 or PR rust-lang#36438. The comment snippet, added by the original commit, has survived the times without any change, becoming outdated at removal of the dedicated pass. Nowadays, grepping for the next_node_id function will show up multiple places in the compiler that call it, but the main rewriting that the comment talks about is still done in the expansion step, inside an innocious looking visit_id function that's called during macro invocation collection.

rust-highfive assigned nrc Sep 13, 2016

jseyfried force-pushed the node_ids_in_expansion branch from 07ce0ae to cb957a7 Compare September 13, 2016 03:46

jseyfried reviewed Sep 13, 2016
View reviewed changes

jseyfried force-pushed the node_ids_in_expansion branch from cb957a7 to 269cd05 Compare September 13, 2016 03:56

jseyfried added 2 commits September 13, 2016 05:11

Avoid needless reexpansions.

50f94f6

Refactor noop_fold_stmt_kind out of noop_fold_stmt.

60440b2

jseyfried force-pushed the node_ids_in_expansion branch from 269cd05 to d3b08e5 Compare September 13, 2016 05:12

eddyb reviewed Sep 13, 2016
View reviewed changes

jseyfried added 2 commits September 13, 2016 05:31

Refactor ExtCtxt to use a Resolver instead of a MacroLoader.

a9821e1

Rewrite the unit tests in ext/expand.rs as a compile-fail test.

20b43b2

jseyfried force-pushed the node_ids_in_expansion branch from d3b08e5 to 6707331 Compare September 13, 2016 05:32

jseyfried added 5 commits September 13, 2016 09:40

Move macro resolution into librustc_resolve.

72a6369

Perform node id assignment and macros_at_scope construction during

c86c8d4

the `InvocationCollector` and `PlaceholderExpander` folds.

Remove scope placeholders from the crate root.

f3c2dca

Expand generated test harnesses and macro registries.

78c0039

Differentiate between monotonic and non-monotonic expansion and

b54e1e3

only assign node ids during monotonic expansion.

jseyfried force-pushed the node_ids_in_expansion branch from 6707331 to b54e1e3 Compare September 13, 2016 09:53

Manishearth mentioned this pull request Sep 15, 2016

Rollup of 9 pull requests #36491

Merged

bors added a commit that referenced this pull request Sep 15, 2016

Auto merge of #36491 - Manishearth:rollup, r=Manishearth

dc75933

Rollup of 9 pull requests - Successful merges: #36384, #36405, #36425, #36429, #36438, #36454, #36459, #36461, #36463 - Failed merges: #36444

bors merged commit b54e1e3 into rust-lang:master Sep 15, 2016

jseyfried deleted the node_ids_in_expansion branch September 16, 2016 20:16

TimNN mentioned this pull request Sep 17, 2016

ICE when macros are used in types. #36540

Closed

jseyfried mentioned this pull request Feb 7, 2017

Tracking issue for "macro naming and modularisation" (RFC #1561) #35896

Closed

est31 mentioned this pull request Nov 6, 2020

The renumber pass is long gone #78795

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Assign node ids during macro expansion #36438

Assign node ids during macro expansion #36438

jseyfried commented Sep 13, 2016 •

edited

Loading

jseyfried Sep 13, 2016

jseyfried commented Sep 13, 2016

eddyb Sep 13, 2016

jseyfried Sep 13, 2016

nrc commented Sep 14, 2016

bors commented Sep 14, 2016

bors commented Sep 14, 2016

bors commented Sep 14, 2016

alexcrichton commented Sep 14, 2016

Assign node ids during macro expansion #36438

Assign node ids during macro expansion #36438

Conversation

jseyfried commented Sep 13, 2016 • edited Loading

jseyfried Sep 13, 2016

Choose a reason for hiding this comment

jseyfried commented Sep 13, 2016

eddyb Sep 13, 2016

Choose a reason for hiding this comment

jseyfried Sep 13, 2016

Choose a reason for hiding this comment

nrc commented Sep 14, 2016

bors commented Sep 14, 2016

bors commented Sep 14, 2016

bors commented Sep 14, 2016

alexcrichton commented Sep 14, 2016

jseyfried commented Sep 13, 2016 •

edited

Loading