Generated textmate grammar highlights punctuation characters as invalid

[iTob191]

A textmate grammar generated by the TextMateHighlightingFragment does not contain a rule for punctuation. As a result, any punctuation characters are picked up by the invalid rule and highlighted accordingly.

I've looked at the code and basically every keyword that does not match \w+ is ignored here and will then be picked up by the invalid rule.

[cdietrich]

@iTob191 can you work on a fix?

[iTob191]

I'm not really sure how to best resolve this.

1. What textmate scope to use? The naming conventions aren't really clear regarding punctuation characters. VS Code uses different scopes for different punctuation characters, for example for typescript: keyword.operator.ternary.ts, punctuation.separator.comma.ts, meta.brace.square.ts, ...
2. Creating a punctuation rule for everything that is not matched here would be a simple solution to prevent valid tokens to be highlighted as errors. But it feels wrong to classify everything that does not match \w+ as punctuation.

For my personal use case, this simple solution with a scope named punctuation.languagename would be sufficient. And I guess it would be sufficient for most languages and certainly an improvement over the current situation.