Rewrite using sublime-syntax format

[yyx990803]

The current highlight is adapted from the ST2 HTML syntax, originally written in the tmlanguage format and compiled from YAML.

ST3 now supports its own sublime-syntax format and ideally this package should be rewritten based on the new HTML syntax in ST3.

This is an open call if anyone wants to take a stab at it - I have included a samples directory to verify basic highlighting behavior. Documentation for ST3 syntax definitions are here.

[skyronic]

I would very much like to contribute to this. Will work on it right now and report back if I could make progress.

[eddyerburgh]

Can we also add support for blocks highlighted as JavaScript by default and blocks highlighted as markdown by default?

[Thom1729]

The current core HTML syntax definition uses the new embed/escape idiom implemented in dev build 3153. It would probably be best to use the same method here. It's not in the syntax definition docs yet, but partial documentation is available here.

Also, I don't know much about Vue in general or this syntax in particular, but if this syntax definition is basically HTML extended with additional features, then it might be possible to use my YAML Macros package to directly extend the core HTML syntax in a forward-compatible manner. That way, the Vue syntax could avoid reimplementing HTML itself. An example (implementing JSX by extending the core JavaScript package) can be seen here.

[skyronic]

@Thom1729 I've made a little bit of progress. I discovered embed/escape while looking at the HTML language but found that it doesn't support with_prototype or rather threw an error while using it directly.

In fact, I tried various methods which worked with less success and am taking this direction right now which seems to be working

%YAML 1.2
---
name: Vue Component
file_extensions: [vue]
scope: source.vue

contexts:
  main:
    - match: '<template>'
      scope: text.html.basic
      push:
        - meta_scope: text.html.basic
        - match: '</template>'
          pop: true
        - include: scope:text.html.basic
        - match: '{{'
          push:
          - meta_scope: source.js
          - match: '}}'
            pop: true
          - include: scope:source.js
    - match: '<style>'
      push:
        - meta_scope: source.css
        - match: '</style>'
          pop: true
        - include: scope:source.css

    - match: '<script>'
      push:
        - meta_scope: source.js
        - match: '</script>'
          pop: true
        - include: scope:source.js

I noticed that we can include: scope:source.js and I found this in the RestructuredText language file. If you're familiar with this, would appreciate some feedback on whether this direction is ok.

[skyronic]

@Thom1729 I probably would clarify that vue template syntax isn't 100% html or JSX.

For example:

  <div>
    {{ foo * 10 + 'hi' }}
    <span
      v-text="foo * 10 + 'hi'"
      :id="foo + 'baz'"
      @click="onClick('hello')">
      Hello
    </span>
  </div>

We need to have some parts as JavaScript like {{ foo * 10 + 'hi' }} and other expressions.

[Thom1729]

The issue with include is that you're relying on the included syntax to pop itself off. In the case of JavaScript, this won't work; the Vue syntax has to force it to pop. This is what caused #107.

The old way of dealing with this is with_prototype. It's the most powerful, but also the most complicated, and Sublime basically compiles an entire copy of the included syntax into yours. (It will always do this if you include.) For Vue, this is a significant downside, because it might want to embed a lot of other syntaxes.

The new way is embed/escape. I think that it should work here. In the case of JavaScript, if you set escape: (?=</script), then you shouldn't need with_prototype. The escape will force the JavaScript contexts to pop when </script is encountered, no matter how deeply nested those contexts are.

[skyronic]

@Thom1729 I actually had some other issues with with_prototype and am not using it right now. Using a technique I discovered in this syntax file.

I don't think we need toextend and add to the JS/CSS/etc, and will definitely switch them to embed. But is there a way to add new matchers/extensions with embed approach?

[Thom1729]

I'm not familiar with reStructuredText, but here's an example of how that could go wrong:

.. raw:: html <style>

span { color: red; }

The remainder of the file will be highlighted as CSS because the <style> tag was never closed. This may be correct for reStructuredText, but it isn't for HTML. Consider the following:

<script>0</script>/

The HTML syntax wants the closing </script> tag to end the JavaScript. But 0</script>/ is perfectly valid JavaScript! And when the parser hits </script>, there will be several JavaScript contexts on the stack above the HTML context that is looking for </script. As a result, bug #107.

But is there a way to add new matchers/extensions with embed approach?

I'm not sure what you mean. Just like the HTML syntax supports embedding several other syntaxes, you can write rules to embed as many syntaxes as you like under basically whatever conditions you like. As a bonus, it doesn't bloat the syntax, and you don't have to mess around with with_prototype. The downside is that the escape rule is "dumb" in the sense that it can't handle escaping or recursion.

[skyronic]

@Thom1729 I've opened a PR #109 please provide comments on whether the approach is ok.

[skyronic]

Marking this issue closed 😄 Closed in #109