Migrate Python to use next-gen scope handlers for text fragment extractors #1862
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
September 7, 2023 12:19
It is not fully working yet due to a bug in the original parser in processSurroundingPairCore() where we get returned a text based instead of tree based element that will be dealt by Pokey Current limitation: '''abc''' with cursor on b and "take string" gives us 'abc' (one quote included only) '''abc''' with cursor on first quote and "take string" gives us '' (first and second quote)
pokey
changed the title
3 tasks
pokey
force-pushed
the
migrate-python-scope-string
branch
from
cfeb2a3 to
5565b66
Compare
|
I have confirmed that with this and #1863 that I can |
github-merge-queue bot
pushed a commit
that referenced
this pull request
Sep 8, 2023
The fallback logic for determining whether to use text-based surrounding pairs or parse-tree-based was broken for text fragment extractors based on next-gen scope handlers (ie using the new `@textFragment` tag in a query file). This breakage meant that in a string like `"""hello"""` in Python, the surrounding pair finder fell back to text-based, which resulted in it thinking `"hello"` was a string, rather than `"""hello"""`, as would happen if it fell back to parse-tree-based This PR fixes the issue by unifying the fallback logic between legacy and next-gen text fragment extractors, so that they don't fall out of sync again See also #1812 (comment); arguably that will make this PR irrelevant, but until then, it's better to have string not be broken when we migrate a language to use `@textFragment` Note that the tests in this PR don't actually test the code yet, because Python is still using legacy text fragment extractors. The tests will start biting when this PR is merged into main and then merged into #1862 - This PR is required by #1862 ## Checklist - [ ] I have added [tests](https://www.cursorless.org/docs/contributing/test-case-recorder/) - [ ] I have updated the [docs](https://github.com/cursorless-dev/cursorless/tree/main/docs) and [cheatsheet](https://github.com/cursorless-dev/cursorless/tree/main/cursorless-talon/src/cheatsheet) - [ ] I have not broken the cheatsheet --------- Co-authored-by: Andreas Arvidsson <andreas.arvidsson87@gmail.com>
pokey
added
lang-python
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
This PR also includes a few additional unit tests for Python strings.