-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
syntax: Keep token span as a part of Token
#61541
Conversation
This comment has been minimized.
This comment has been minimized.
c380fa5
to
e79d91a
Compare
This comment has been minimized.
This comment has been minimized.
r=me with |
☔ The latest upstream changes (presumably #57428) made this pull request unmergeable. Please resolve the merge conflicts. |
⌛ Testing commit 3a31f06 with merge 03d5df4e519af16adb22146dbccabfda5f4db633... |
💔 Test failed - checks-travis |
The job Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
The rand crate broke. |
syntax: Keep token span as a part of `Token` In the world with proc macros and edition hygiene `Token` without a span is not self-contained. In practice this means that tokens and spans are always stored and passed somewhere along with each other. This PR combines them into a single struct by doing the next renaming/replacement: - `Token` -> `TokenKind` - `TokenAndSpan` -> `Token` - `(Token, Span)` -> `Token` Some later commits (fb6e2fe and 1cdee86) remove duplicate spans in `token::Ident` and `token::Lifetime`. Those spans were supposed to be identical to token spans, but could easily go out of sync, as was noticed in #60965 (comment). The `(Token, Span)` -> `Token` change is a soft pre-requisite for this de-duplication since it allows to avoid some larger churn (passing spans to most of functions classifying identifiers).
☀️ Test successful - checks-travis, status-appveyor |
parser: Remove `Deref` impl from `Parser` Follow up to rust-lang#61541 You have to write `self.token.span` instead of `self.span` in the parser now, which is not nice, but not too bad either, I guess. Not sure. Probably still better than people using both and being confused about the definition point of `span`. r? @oli-obk @estebank
parser: Remove `Deref` impl from `Parser` Follow up to rust-lang#61541 You have to write `self.token.span` instead of `self.span` in the parser now, which is not nice, but not too bad either, I guess. Not sure. Probably still better than people using both and being confused about the definition point of `span`. r? @oli-obk @estebank
parser: Remove `Deref` impl from `Parser` Follow up to rust-lang#61541 You have to write `self.token.span` instead of `self.span` in the parser now, which is not nice, but not too bad either, I guess. Not sure. Probably still better than people using both and being confused about the definition point of `span`. r? @oli-obk @estebank
parser: Remove `Deref` impl from `Parser` Follow up to rust-lang#61541 You have to write `self.token.span` instead of `self.span` in the parser now, which is not nice, but not too bad either, I guess. Not sure. Probably still better than people using both and being confused about the definition point of `span`. r? @oli-obk @estebank
syntax: Remove `Deref` impl from `Token` Follow up to rust-lang#61541 r? @oli-obk
} | ||
} | ||
|
||
impl PartialEq<TokenKind> for Token { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This impl is technically "wrong": it's not symmetric, and, if we made it symmetric, it would not be transitive.
I've tried to remove it, but got a dozen of screen-fulls of errors :D So I guess it's better to leave it as is: seems harmless, even if surprising a bit.
In the world with proc macros and edition hygiene
Token
without a span is not self-contained.In practice this means that tokens and spans are always stored and passed somewhere along with each other.
This PR combines them into a single struct by doing the next renaming/replacement:
Token
->TokenKind
TokenAndSpan
->Token
(Token, Span)
->Token
Some later commits (fb6e2fe and 1cdee86) remove duplicate spans in
token::Ident
andtoken::Lifetime
.Those spans were supposed to be identical to token spans, but could easily go out of sync, as was noticed in #60965 (comment).
The
(Token, Span)
->Token
change is a soft pre-requisite for this de-duplication since it allows to avoid some larger churn (passing spans to most of functions classifying identifiers).