Using tree-sitter-highlight library from upstream

[jspawar]

Hello, new to a lot of this, so apologies in advance for any misunderstandings of course. Was poking around and from what I appear to see is that the highlighting provided by this module seems to be done a bit "manually" by something like querying the tree-sitter "tree" for a buffer and using that to highlight?

Was curious then why tree-sitter-highlight wasn't used instead? (Assuming it wasn't used because it's not in the Cargo.toml)

I also tried toying around with that upstream library and seems plenty sufficient (so far as I understand it is what's actually intended to back highlighting in the bigger consumers of tree-sitter such as Atom or GitHub itself). Gave it some input files and it correctly provided region ranges along with the correct type of node/token.

Wonder if it could also mitigate some issues folks are having such as #94 and #76? The output I received was able to correctly identify local variables, so seems like it should?

[ubolonton]

Was curious then why tree-sitter-highlight wasn't used instead? (Assuming it wasn't used because it's not in the Cargo.toml)

That library is designed for the use case of highlighting static sources, without handling edits. The underlying mechanism is the same: querying the syntax tree using a Lisp-like query language.

I also tried toying around with that upstream library and seems plenty sufficient (so far as I understand it is what's actually intended to back highlighting in the bigger consumers of tree-sitter such as Atom or GitHub itself).

GitHub's use case is the static highlighting mentioned above.
Atom doesn't use that library. Its highlighting mechanism is more ad-hoc, which predates the Lisp-like query language, and hasn't been updated for a long time AFAICT.

Gave it some input files and it correctly provided region ranges along with the correct type of node/token.

Wonder if it could also mitigate some issues folks are having such as #94 and #76? The output I received was able to correctly identify local variables, so seems like it should?

The hard part is handling the local predicates in an efficient manner, in the presence of (dynamic) text edits. That library only handles the static case, so it won't help. The same applies to other contextual concerns, like injected language blocks.