Publish on NPM (for use in Atom grammars)

[turbo]

Please add a CSON which defines the language and file recognition, so that this grammar can be used in Atom.

[turbo]

Actually, I seem to have misread the documentation. All that would be needed for me to use this repo in an Atom grammar is for it to be available on npm. Now, I can't find it there. So the request changes to: Publish this package on npm :).

[sogaiu]

FWIW, I think npm also supports dependences specified via git urls: https://docs.npmjs.com/files/package.json#git-urls-as-dependencies

I've been using this approach for things in development.

Here's a concrete example: https://github.com/sogaiu/vscode-clojure-defs/blob/557b8e4c5546a6d057c8095f10b00c5416f19701/package.json#L33

[sogaiu]

On a side note, here's an example .cson: https://github.com/sogaiu/language-clojure/blob/e9f720c1f4a793322b103b8b4ce646d11b35d726/grammars/tree-sitter-clojure.cson

In case someone wants to make an attempt :)

[turbo]

@sogaiu I just remembered that you can actually just push the package to GitHub, as GitHub now supports packages (see sidebar right). But since GitHub now own npm, I guess they'll eventually converge.

[sogaiu]

@turbo You may be right :)

BTW, as the author of a grammar for another lisp-like language, I want to mention that it has turned out to be unusually tricky to get things to work well. Especially once one begins testing on real-world code certain things may start turning up that may motivate one to want to modify one's grammar and in some cases forgo supporting certain constructs for the sake of correctly supporting more important constructs.

I don't know what TravonteD's take on the current situation is, but in the few grammars I worked on this was the recurring pattern. So this is an attempt at a "please don't be surprised if things change" statement :)

[turbo]

Debugging tree-sitter grammars is a bit outside of my parsing comfort zone, but the tests look somewhat comprehensive. I'm probably going to work on PEG or ST4 grammars because those are ones I can use, in editors I use ;).

[sogaiu]

@turbo I haven't used SublimeText though I keep hearing good things about it (I presume ST4 means SublimeText 4). I think I saw an extensive thread in tree-sitter's repository issues with a disucssion that was way over my head that discussed SublimeText's approach (I guess there have been multiple).

I've been a happy user of PEGs though especially in the Janet programming language.

Regarding testing, I ended up not using the built-in system for Clojure because I didn't find it to be helpful enough (ofc, better to have some testing than none!) and it was awkward to use at scale.

For the 3 grammars I've worked on I have each parser work on as many real-world samples as I can find.

You can see a sample invocation here: #5

On a side note I'm currently writing property-based tests for a Clojure grammar using the hypothesis testing framework and the py-tree-sitter bindings: https://github.com/sogaiu/prop-test-ts-clj

This approach is helping me to observe certain things about the grammar that I had failed to see (and I've made some modifications as a result). I think this is partly because I'm now a user of the grammar in a programmatic context -- dogfooding FTW!

I hope TravonteD will forgive these "extra" comments :)

[turbo]

@sogaiu Long story short about ST4 and the new sublime-syntax format is that it's now able to parse pretty much all advanced constructs other next-gen systems like tree-sitter can parse. There was heated discussion because the tree-sitter creator said some questionable things about the ST4 parser capabilities. I'm just happy there are now more next-gen options.

PEGs are used extensively in TextAdept grammars, even the Rust one is fairly simple: https://github.com/leafo/lua-syntaxhighlight/blob/master/syntaxhighlight/textadept/rust.lua

[sogaiu]

@turbo Thanks for the elaboration.

I agree it's nice that there are significantly better options than before.

Thanks for the concrete example -- I only recognize things like Cmt because Janet borrowed them :)