Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update languages.toml: tree-sitter-lua grammar #9727

Merged
merged 3 commits into from
Feb 28, 2024

Conversation

briandorsey
Copy link
Contributor

repo has moved, use new URL and the rev of the latest release (v0.0.19)

There is a more recent release of the tree-sitter-lua grammar (and new repo url, which appears to be the same repo, just moved):
tree-sitter-grammars/tree-sitter-lua@88e4464

I'm a first-time contributor, opening this PR per request from @archseer on #9726.

I tested locally by adding a .config/helix/languages.toml file with the following content (the source = line is the same content as this PR):

use-grammars = { only = [ "lua" ] }

[[grammar]]
name = "lua"
source = { git = "https://github.com/tree-sitter-grammars/tree-sitter-lua", rev = "88e446476a1e97a8724dff7a23e2d709855077f2" }

Then updated the grammar:

% hx --grammar fetch
Fetching 1 grammars
1 updated grammars
	lua now on 88e446476a1e97a8724dff7a23e2d709855077f2
% hx --grammar build
Building 1 grammars
1 grammars built now
	["lua"]

Then confirmed that the new grammar is being used. It is, yea!

fixes: #9726

repo has moved, use new URL and the rev of the latest release (v0.0.19)
@woojiq
Copy link
Contributor

woojiq commented Feb 26, 2024

Very often, when you update a version of the Tree-sitter grammar, the node names change, causing the current helix queries to become inaccurate. Check the changelog and update our queries accordingly.
For example, I see that their highlight queries have changed several times in the last 2 years: https://github.com/tree-sitter-grammars/tree-sitter-lua/commits/main/queries/highlights.scm

@the-mikedavis the-mikedavis added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. A-language-support Area: Support for programming/text languages labels Feb 27, 2024
@briandorsey
Copy link
Contributor Author

Are there any tools or docs to help validate that changes to these queries correctly map to helix expectations? I assume it's not ok to just copy the query files over from the tree-sitter-lua repo, correct?

I spent about an hour trying to learn about this. I started by poking around at the files in the tree-sitter-lua repo vs. what is in the helix repo... they are very different, and I don't really know where to start changing them with any confidence that it wouldn't break something. Later, started looking at tree-sitter docs and trying to run tree-sitter query (some progress) or playground (blocked on wasm/Docker troubleshooting)... and then ran out of the time I have available to look at it today.

At the moment, this looks like it'll take a while to learn enough to update the queries with any confidence.

@the-mikedavis
Copy link
Member

The queries that are in that repo are ok to copy but they'll need some small modifications. We use a custom set of captures https://docs.helix-editor.com/master/themes.html#syntax-highlighting, so @conditional for example should become @keyword.control.conditional, @number should become @constant.numeric, etc.

We will also need to re-order the patterns a little because neovim currently has the opposite precedence compared to us (though that will change with #9436). So this identifier pattern needs to move to basically the bottom of the file: after all other uses of (identifier). Also this function pattern should move above all of the other patterns that highlight functions.

In general you can use :tree-sitter-subtree on a selection to see what the tree looks like and then write queries based on that

a novice attempt to port query updates from the
source repo to Helix captures and ordering
@briandorsey
Copy link
Contributor Author

Thanks for the guidance! I attempted to follow it. I'm not very confident that I got the ordering or captures correct.

The :tree-sitter-* commands were very helpful for visualizing/understanding what's happening with the mapping.

Aside: I was a bit confused how the :tree-sitter-* commands decided what to show. -subtree seemed to perhaps parse the whole file when a full line was selected (with x)

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
Copy link
Member

@the-mikedavis the-mikedavis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@the-mikedavis the-mikedavis added S-waiting-on-review Status: Awaiting review from a maintainer. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 28, 2024
@the-mikedavis
Copy link
Member

-subtree seemed to perhaps parse the whole file when a full line was selected (with x)

:tree-sitter-subtree tries to find the largest node that fully contains the current selection. Usually newlines are part of the root of the syntax tree so :tree-sitter-subtree on a full line gives the root. You can use trim_selections (_) before to trim the leading/trailing whitespace of the selection

@pascalkuthe pascalkuthe merged commit f03b91d into helix-editor:master Feb 28, 2024
6 checks passed
@briandorsey briandorsey deleted the tree-sitter-lua-grammar branch February 28, 2024 20:03
postsolar pushed a commit to postsolar/helix that referenced this pull request Apr 4, 2024
* update languages.toml: tree-sitter-lua grammar

repo has moved, use new URL and the rev of the latest release (v0.0.19)

* update highlight queries

a novice attempt to port query updates from the
source repo to Helix captures and ordering

* Apply suggestions from code review

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>

---------

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
mtoohey31 pushed a commit to mtoohey31/helix that referenced this pull request Jun 2, 2024
* update languages.toml: tree-sitter-lua grammar

repo has moved, use new URL and the rev of the latest release (v0.0.19)

* update highlight queries

a novice attempt to port query updates from the
source repo to Helix captures and ordering

* Apply suggestions from code review

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>

---------

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
Vulpesx pushed a commit to Vulpesx/helix that referenced this pull request Jun 7, 2024
* update languages.toml: tree-sitter-lua grammar

repo has moved, use new URL and the rev of the latest release (v0.0.19)

* update highlight queries

a novice attempt to port query updates from the
source repo to Helix captures and ordering

* Apply suggestions from code review

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>

---------

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
smortime pushed a commit to smortime/helix that referenced this pull request Jul 10, 2024
* update languages.toml: tree-sitter-lua grammar

repo has moved, use new URL and the rev of the latest release (v0.0.19)

* update highlight queries

a novice attempt to port query updates from the
source repo to Helix captures and ordering

* Apply suggestions from code review

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>

---------

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-language-support Area: Support for programming/text languages S-waiting-on-review Status: Awaiting review from a maintainer.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Consider update from old Lua grammar
4 participants