Support evolving Golang syntax by switching grammar repo

[flowchartsman]

The Go grammar for Vscode is effectively frozen at this time since the atom syntax it is built from is no longer maintained. To my knowledge, the only real change to go syntax highlighting that this affects is for the new print verb %w introduced in Go1.13, but there will definitely be more. I would support VScode switching to a community-maintained fork such as this one or forking and maintaining their own until such time (hopefully) as tree sitter is a thing and we can resume "borrowing" from Atom once again ;)

[Jacalz]

Could this also be the reason for the issues that I reported in microsoft/vscode-go#2892? If that is the case, then I would really love to see this being fixed. Only thing keeping me from switching from Atom at the moment.

[flowchartsman]

@Jacalz If it involves the grammar, then yes. The grammar for Go is effectively frozen at the moment, since it's derived from the old Atom grammar, which is now deprecated and not accepting any PRs.

That said, it's not as simple as enabling a new repo, since whatever definitions allow Atom to highlight the way you describe are almost certainly implemented in tree-sitter, so they'd have to be backported to the first-mate syntax VSCode currently uses.

[Jacalz]

Thanks for the information. Hopefully this will be fixed sooner rather than later because it is definitely holding back my switch to vscode and I could see it doing the same for some others too.

[alexr00]

@jeff-hykin, Your repo https://github.com/jeff-hykin/better-go-syntax/ is mentioned at the top of this issue. Will it be getting the same TLC as the C++ grammar?

[jeff-hykin]

Sadly, I won't be able to put in the amount of effort like I have for C++. With that said:

Most TM grammar repos are simply a .json, .plist, or .yaml file and https://github.com/jeff-hykin/better-go-syntax/ would at least be that. The Go syntax is fundamentally and vastly more simple than C++, so I expect it will not need nearly as much attention. The Go syntax is 600 lines while the C++ syntax is 19,000 lines. While at the moment I can't create advanced new features, I can approve merge requests and do basic fixes on a weekly or monthly timeframe to prevent the Go grammar from stagnating.

Switching would certainly net positive, however it would not be anything close to the level of improvement C++ has seen.

If switching right now, the Go syntax

• wouldn't have unit tests
• wouldn't have integration tests
• would not be generated by a scripting language (which, in my opinion, is necessary for long term maintenance of any TM grammar)

Those things will be possible once Matter123 and I get the Textmate grammar library finalized, which could possibly be done the end of winter break. I likely still won't be able to do advanced feature additions, but the tests and scripting language will make it very easy for outsiders to contribute in a safe and reliable way.

NOTE: If you do decide to switch, the main syntax file is source/generated.tmLanguage.json. The syntax file being referenced inside the package.json (for the extension) is part of an old file structure that will need to be changed once the textmate grammar library is finalized.

[alexr00]

That does sound like a net positive, thanks for the details. VS Code has it's own integration tests for each language, so there will be some coverage for now. I'll look at switching in December.

[alexr00]

@jeff-hykin I've pulled in your go grammar. The diff between it and the old one are pretty minimal, so it seems low risk right now. We will likely release the next version of VS Code at the end of January or beginning of February 2020, so there's plenty of time to revert if you're not ready for it to go out in the release!

[Jacalz]

I will do some testing with those changes to see if they fix my issues. Thanks for pulling in that better syntax highlighting 🙂

[jeff-hykin]

Thanks @alexr00 ! Just wanted to let you know I cleaned up the go syntax repo today so its ready for contributions.

There's already a few pre-existing hard to fix bugs (a performance bug) that I won't be able to put in the time to address. But I can guide people to fix them and verify their fixes which should keep the syntax from stagnating.

[ramya-rao-a]

@alexr00 Is the associated change here part of the January release or available only in the current Insiders?

[alexr00]

Currently available only insiders, but will be going to stable in the January release.

[isavcic]

Note that, compared to Sublime, VS Code Go syntax highlighting is lacking. Notice the function signatures. You could find more examples of methods etc. being treated as normal text instead of "special".

[Jacalz]

I totally agree with you @isavcic, vscode is very lacking sadly :(

[jeff-hykin]

@isavcic any idea where Sublime is getting it's syntax files? Its likely we can pull in their changes if they're using open source code.

I remember editing the syntax files in Sublime several years ago, but after downloading Sublime and digging through the files I can't find the source of their Go syntax. The filename should be Go.sublime-syntax.