Need a better grammar support for Rust lang

[zimond]

This problem is raised up for several times before (#49932, #45628) But not resolved. VSCode team member replies to these issues as: We are using the same grammar with Atom. In fact, Atom has already changed to tree-sitter syntax as the bundled Rust language grammar support package (https://github.com/atom/atom/tree/master/packages/language-rust-bundled).

However, Vscode's bundled rust support extension is based on Zargony's package which is not updated for nearly a year and many issues open with no reply at all.

Currently every highlighting problem in #49932 still happens and that's not simply a "theme" thing as @mjbvz mentioned in the orignal issue, I think.

In the original issue, @mjbvz also declares that

Since we use the same grammar, I believe this is just a matter of theming. As far as I can tell, the atom rust extension injects its own themeing rules to provide better colors.

the plugin he referred to is not even Zargony's package. it's a package from @miqh, who also ported the atom package to vscode, and fails to correctly provide grammar for Rust as well.

Hope that I get everything right, thanks for any help.

[takkuumi]

me too

[alexr00]

As @zimond said and @mjbvz said in #45628, we get our rust grammar from the community. If we tried to provide grammars and theming for every language ourselves then the support would be much worse since we aren't experts in all languages 😊 If there is a better, well supported, TM grammar that we can pull in for rust then I'm happy to pull it in and try it out.

For additional theming, this is something that an extension could do.

Since all we can do here is pull in a different grammar, I'm going to mark this as a duplicate of #45628.

[vscodebot]

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for existing issues here. See also our issue reporting guidelines.

Happy Coding!

[zimond]

This is ridiculous. The bundled rust syntax is not only incomplete, but also incorrect (Check this issue as a crucial bug), with many keywords missing. If you insist that vscode team is not capable of "provide grammars and theming for every language ourselves", then please just remove the bundled package.

And I would like to state it one more time: This is not about "theming". Just mark as duplicate could not make any improvement.

You know that current grammar is incorrect, and could be improved. You cannot do it, that's Ok. But closing issues will just silence the problem and prevent more works to be done (either by the community or by vscode team). That's simply not helping.

[zimond]

I believe that this issue is in fact related to #50140, not just a textmate grammar or theming problem.

[alexr00]

@zimond we use the *extension-candidate label to keep track of closed issues that would be best solved by someone writing and maintaining an extension. Since this issue covers both theming (extension solvable) and grammar (also techinically solvable with an extension, but we do provide it inbox) I'll reopen it for the grammar.

If I thought I could get away with just removing the built in rust extension then I would. However, there are people out there who are using it so I can't just take it away from them even if it has flaws.

[alexr00]

Given the community feedback that VS Code's rust grammar has problems, we're looking for a more frequently maintained TM rust grammar. If anyone is interested in contributing a new grammar for rust please reach out!

[takkuumi]

see this #45628 and this #42145 ， They didn't take it seriously @alexr00 @zimond

[zimond]

@NateLing No need to push it this hard. I think the lack of correct syntax highlighting is due to the limitation of TextMate's grammar format. Now vscode team is thinking about using visual studio's grammar as an upgrade. I think all we could do is wait for it, or maybe tree-sitter support.

[Geobert]

There are so many issues with highlighting in so many languages that can be solved by using tree-sitter, I don't get the mentality of keeping TM which is said to be hard, limited etc.

[georgewfraser]

I've added support for tree-sitter via an extension: https://marketplace.visualstudio.com/items?itemName=georgewfraser.vscode-tree-sitter

Rust is supported using the same grammar as Atom. Tree-sitter coloring is on the right:

[zimond]

@georgewfraser you are an angel

[jk-gan]

@georgewfraser Thanks for that!

[Geobert]

There is another extension base on tree-sitter with Rust support as well:
https://github.com/EvgeniyPeshkov/syntax-highlighter/

I'm not the author, but I've helped him to improve the highlighting by giving him as much info as possible on Rust (as he doesn't know much about Rust). Still some issues in discussion, but out of the box, it does a pretty good job.

[dustypomerleau]

@alexr00 I have released an improved TextMate grammar via the Rust Syntax extension. I am happy to submit a PR to make this the default in VS Code. The current arrangement, where code comments indicate that the Atom grammar is upstreamed (and therefore PRs should be submitted to Atom instead) I think discourages people from trying to get grammar improvements merged. Given the size of the VS Code community at this point, would you be happy to accept direct PRs on grammars?

[alexr00]

@dustypomerleau allowing direct PRs on the grammars means that we are accepting maintenance responsibility for the grammar, which isn't feasible given the size of the team and the diversity of languages we have grammars for.

I am more than happy to move away from the Atom grammar though. If the Rust Syntax extension provides a grammar file that VS Code can consume, please do provide a pointer to it and I can try pulling it into VS Code and we can see what others think of the new grammar for a month or so.

[dustypomerleau]

@alexr00 That would be awesome—the generated JSON is in this file. I appreciate your time.

[alexr00]

@dustypomerleau I adopted your grammar in a PR and I have some questions in there: #108254

[alexr00]

For everyone who has been following this issue, dustypomerleau/rust-syntax is the default Rust grammar for VS Code now. Closing since this issue has been satisfied.

[alexr00]

Verification is accomplished through the grammar colorization tests.