Additional support for sublime-syntax grammar

[gandm]

Sublime text 3 has implemented a new grammar format that seems much better than the traditional textmate grammar. Is there any chance this can be implemented?

It is described here https://www.sublimetext.com/docs/3/syntax.html

[Alhadis]

Seems to be a duplicate of #36.

[winstliu]

Thanks @Alhadis.

[gandm]

No it's not!

[Alhadis]

How? It seems to be syntactic sugar around the standard TextMate-grammar parsing.

[gandm]

No it's has a concept of state.

[Alhadis]

What do you mean, a concept of state?

[gandm]

For example you can match a single char, say a + sign and then follow this by pushing a number of contexts onto the stack. E.g. variable name, numeric number,... only one will pop the stack. This is hard in normal grammar as the only code that spans multiple lines is begin/end and you have to know what ends it before you start.

[gandm]

Having said that, I am not too bothered if the issue is closed.

[Alhadis]

It's possible through nested patterns, although ugly. So... theoretically, that's still syntactic sugar over what's already possible now.

[winstliu]

Thanks for explaining further @gandm - perhaps I should actually scroll down the link next time. There is also variable support, which I don't think CSON has, so I'm going to re-open this.

[gandm]

All I can say is this would have made my life easier when I wrote language-babel

[borela]

I wouldn't consider "just a syntax sugar" when it makes writing syntaxes a way easier than the traditional tmLanguage. Also you can have different control paths with push vs set and include external files which is perfect when dealing with lists of words like so: https://github.com/borela/naomi/blob/master/syntaxes/fjsx15/reactjs.sublime-syntax

[gandm]

tree-sitter soon available so closing.

[borela]

@gandm Can the sublime-syntax be converted to tree sitter?

[Arcanemagus]

For reference, @borela's question is answered in tree-sitter/tree-sitter#139 😉.

[borela]

@Arcanemagus So far arguments in favor of tree sitter I heard only apply to limitations in the tmLanguage, you can assign the correct meaning to tokens using sublime syntax.

[gandm]

I think tree-sitter is vastly superior. Fast, accurate with additional intelligence. You cannot create a true language parser using sublime-syntax nor textmate. Certainly not JavaScript.

[Arcanemagus]

@borela There is a major difference between a true parser such as what tree-sitter provides, and the attempts to hack together a similar result as is done in TextMate grammars... or Sublime Text's new format. I'd recommend reading up on parsers of context free grammars and if you still have questions feel free to follow up over in tree-sitter/tree-sitter#139.

[borela]

@Arcanemagus that's what I was not getting, to me looked like tree sitter was just generating a tree which could be done by analyzing the scopes generated by the sublime syntax.

It didn't occur to me that tree sitter was also parsing.

[Alhadis]

Certainly not JavaScript.

... lol, what the hell.

[borela]

@gandm You can assign the correct scopes using sublime syntax and even distinguish whether /abc/ is a regex literal or something divided by abc depending on whats before it, I did exactly that on https://github.com/borela/naomi, other packages does the same, the only limitation is the per line regex, but that's sublime's limitation.

But as I talked to Max, he did convinced me to use Tree Sitter for more complex languages (C++, MQL5, etc...) because I often have to deal with badly formatted code that spans across multiple lines.