-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TokenMap -> SpanMap rewrite #15959
TokenMap -> SpanMap rewrite #15959
Conversation
d7fd984
to
8707c13
Compare
aec29d9
to
98cfdde
Compare
e4f26d6
to
c57fd9d
Compare
c57fd9d
to
7a8c4c0
Compare
d2fab1a
to
f48fa0c
Compare
9ca83b8
to
7140106
Compare
7140106
to
0003e56
Compare
7c6897f
to
5edf7bd
Compare
e952e18
to
02a3a94
Compare
Okay this is ready now, so we can merge this after tomorrows release, and depending on the fallout we might want to skip the following stable release, though I don't expect this to be that problematic. |
12fefc3
to
05f4e55
Compare
05f4e55
to
18f1a3c
Compare
@bors r+ |
📋 Looks like this PR is still in progress, ignoring approval. Hint: Remove [WIP] from this PR's title when it is ready for review. |
@bors retry |
@bors r+ |
☀️ Test successful - checks-actions |
resolver, | ||
) | ||
}; | ||
let err = parse_err.or(err); | ||
if cfg!(debug) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(sorry for the duplicate ping, I'm not sure how well they work in commit comments)
@Veykril is this supposed to be cfg!(debug_assertions)
or can we call it unconditionally?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it was supposed do be debug_assertions
(cause it may reallocate unnecessarily)
Opening early so I can have an overview over the full diff more easily, still very unfinished and lots of work to be done.
The gist of what this PR does is move away from assigning IDs to tokens in arguments and expansions and instead gives the subtrees the text ranges they are sourced from (made relative to some item for incrementality). This means we now only have a single map per expension, opposed to map for expansion and arguments.
A few of the things that are not done yet (in arbitrary order):
- [ ] remove all uses of dummy spans (or if that does not work, change the dummy entries for dummy spans so that tests will not silently pass due to havin a file id for the dummy file)macro_expand
, the sole caller of it isparse_macro_expansion
, and both of these are lru-cached with the same limit so having it be a query is pointless()
expression wrapping in expansions / alternatively prioritize getting invisible delimiters working again- [ ] span crate containing all the file ids, span stuff, ast ids. Then remove the dependency injection generics from tt and mbeFixes #10300
Fixes #15685