Rewrite to operate on tokens #142
Comments
|
I'm helping @alexcrichton with modifying the parser to run on TokenStream objects. My current unrebased branch is https://github.com/mystor/syn/tree/proc-macro2. |
|
I love both of you. @mystor question about |
|
@dtolnay Yes, As you can probably tell from my current WIP branch, I was trying to do the minimal number of changes to the rest of |
This ended up being a bit larger of a commit than I intended! I imagine that this'll be one of the larger of the commits working towards dtolnay#142. The purpose of this commit is to use an updated version of the `quote` crate which doesn't work with strings but rather works with tokens form the `proc-macro2` crate. The `proc-macro2` crate itself is based on the proposed API for `proc_macro` itself, and will continue to mirror it. The hope is that we'll flip an easy switch eventually to use compiler tokens, whereas for now we'll stick to string parsing at the lowest layer. The largest change here is the addition of span information to the AST. Building on the previous PRs to refactor the AST this makes it relatively easy from a user perspective to digest and use the AST still, it's just a few extra fields on the side. The fallout from this was then quite large throughout the `printing` feature of the crate. The `parsing`, `fold`, and `visit` features then followed suit to get updated as well. This commit also changes the the semantics of the AST somewhat as well. Previously it was inferred what tokens should be printed, for example if you have a closure argument `syn` would automatically not print the colon in `a: b` if the type listed was "infer this type". Now the colon is a separate field and must be in sync with the type listed as the colon/type will be printed unconditionally (emitting no output if both are `None`).
This ended up being a bit larger of a commit than I intended! I imagine that this'll be one of the larger of the commits working towards dtolnay#142. The purpose of this commit is to use an updated version of the `quote` crate which doesn't work with strings but rather works with tokens form the `proc-macro2` crate. The `proc-macro2` crate itself is based on the proposed API for `proc_macro` itself, and will continue to mirror it. The hope is that we'll flip an easy switch eventually to use compiler tokens, whereas for now we'll stick to string parsing at the lowest layer. The largest change here is the addition of span information to the AST. Building on the previous PRs to refactor the AST this makes it relatively easy from a user perspective to digest and use the AST still, it's just a few extra fields on the side. The fallout from this was then quite large throughout the `printing` feature of the crate. The `parsing`, `fold`, and `visit` features then followed suit to get updated as well. This commit also changes the the semantics of the AST somewhat as well. Previously it was inferred what tokens should be printed, for example if you have a closure argument `syn` would automatically not print the colon in `a: b` if the type listed was "infer this type". Now the colon is a separate field and must be in sync with the type listed as the colon/type will be printed unconditionally (emitting no output if both are `None`).
Using rust-lang/rust#40939 and
proc-macro2. @alexcrichton is working on this.The text was updated successfully, but these errors were encountered: