Universal Language Injections

[felixfbecker]

Language injections is a feature of IDEs like Webstorm and PHPStorm that auto-detects when a language is use inside another language and then shows syntax highlighting for that injected language. VS Code currently supports this in a few places:

• <style> tags in HTML get CSS highlighting and IntelliSense
• <script> tags in HTML get JS highlighting and IntelliSense
• HTML in a PHP file outside of <?php ?> gets highlighting but no IntelliSense

But there are many other contexts where this would be helpful, that are currently not supported and impossible to all hardcode:

• style attribute in HTML contain CSS
• String literals in languages like PHP often contain SQL, see [php] highlighting for mysql in php file #1361. PHPStorm supports this for PHP, but Webstorm for example does not support this for NodeJS. It should really be supported in any language.
• String literals commonly contain HTML in Javascript or PHP (e.g. with jQuery, Angular or Vue templates)
• String literals commonly contain GraphQL queries in JavaScript
• HTML event attributes contain Javascript, like onclick="event.preventDefault()"
• Countless HTML attributes in the Angular framework have Javascript-like expressions in them, like ng-if="ctrl.items.length > 0 && someOtherValue", but also the mustache syntax {{ ... }}.
• <script type="text/ng-template"> contain Angular templates which is nothing but HTML. This pattern of templates in script tags exists in many of the major client side MVC frameworks.
• Javascript has RegExp literals. In other languages like PHP, RegExps are passed as strings. This is also the case in tasks.json problem matchers for VS Code (RegExp inside JSON string)

All of these examples currently don't get colored and receive no autocompletion. It would be awesome if VS Code was able to automatically detect languages in any context (like string literals, HTML attributes and tags) by some language patterns, like < > tags for HTML/XML, keywords like SELECT, INSERT, etc. for SQL or template string tags like gql for GraphQL. This could be contributed by extensions or language servers.

This would be one of the best features VS Code could offer.

[bpasero]

@egamma @dbaeumer @aeschli fyi

[felixfbecker]

Added RegExp example

[farfromunique]

Piggybacking: Autocomplete for the injected language would be awesome! It's really disengaging to be typing HTML for output from PHP and have PHP syntax suggestions.

[anyong]

Would love to have SQL syntax highlighting throughout my projects. Currently I always write my SQL in a separate file with syntax highlighting, then copy it into my project where it needs to go. In VS Code it's just highlighted as a normal string, so very hard to catch errors and whatnot.

[ricardobeat]

Javascript and CSS highlighting inside <script> and <style> tags should be a basic feature.

[ciel]

I love this idea. Lack of template coloring is one of the main reasons I kept using WebStorm over Visual Studio Code for some projects.

[MarkPieszak]

I don't know if this would help in VSCodes case, but GregOnNet came up with a brilliant solution for adding HTML highlighting to the back-tick section here for Atom-typescript

TypeStrong/atom-typescript#948

Hoping that might help potentially add this ability to VSCode!

[felixfbecker]

@mahmoudymy I don't think is a good solution to hardcore template support for every framework out there. I want syntax highlighting in all kinds of template strings, in my gulp tasks, in my SQL queries, ...

[polina-c]

How about Mason: perl inside html?

[mikeal]

would love this! :)

[MattMcFarland]

Please add this! I have been working on a choo framework project and because it uses es6 template tags for html and then even the tags inside can be escaped it just turns into a mess with syntax highlighting. I think I cant find anything that works other than jetbrains language injection. Do they have a patent on that?

[MarkPieszak]

@MattMcFarland Atom handles es6 template strings really well at the moment as well, you could use that in the meantime until it's added to Code.

[MattMcFarland]

@MarkPieszak Yes I've tried atom, and it doesnt handle them well enough, any javascript code that is injected for onclick events for example atom thinks is a string. Also FWIW you can use atom grammar in vscode and get similar results.

[a-s-o]

Hello everyone,

I was just wondering whether a plugin can accomplish this. My pitch is to have a language specific plugin that does the injections based on its own settings. For instance, a plugin for js could perform injection based on tagged string templates with the following user configurable settings:

Tag         |  Language
-------------------------------
html        | HTML
css         | CSS
graphql     | GraphQL

Usage would be as follows:

import html from 't7';
import css from 'csjs';
import graphql from 'graphql-tag';

export const TodoComponent = {
  template: ({ styles }) => html`
    <div class=${styles.todoApp}>
      <Header>
        <NewTodo class=${styles.newTodo} />
      </Header>
      <ul>
        ${Todos}
      </ul>
    </div>
  `,

  styles: css`
    .todoApp {
      background: yellow;
    }

    .newTodo {
      display: flex;
    }
  `,

  queries: graphql`
    query getTodos ($user: String) {
      todoList (user: $user) {
        id
        title
        completed
      }
    }
  `
};

So basically, users configure the tag they want to match and are then are require to properly name those tags in scope to identify the syntax to be injected.

Is this theoretically possible in a vscode extension? Or would this require editor level changes? I don't have experience writing a plugin yet, but I am willing to take a stab if possible.

[onetrev]

While Universal Language Injections would be amazing, for PHP at least @kanlukasz you can get Intellisense!

This extension (PHP Intelephense), since it's more recent updates, works perfectly for HTML in PHP! Definitely works better than other PHP Intellisense options I've tried. I'm not entirely sure how the creator (Ben Mewburn) got around the limitations but I'm glad he did.

[erichiller]

Also, for Markdown this would be very much appreciated. (as in other languages inside markdown files using triple backtick)

[Logerfo]

AFAIK this already works for markdown because it's implemented in the markdown language engine for vscode

[michaelblyons]

@Logerfo I know this thread is very long, but I have written about the differences before which you can find if you look for them.

Also, to everyone else: Please stop adding "+1"-like comments. Upvote the issue and relevant comments you agree with, but only post a comment yourself if there's something new to add. That will shorten the cognitive load for people to scan the whole thread.

[vadimmos]

Inside html, between <script></script> tags, js is supported at a very low level. JsDoc doesn't work here, and there are no hints about js in linked scripts. Inside js, there is no support for anything else at all. Support for one code within another code is a very necessary technology, especially in the web.

[leeoniya]

well, the good news is that CodeMirror 6 just entered beta:

https://marijnhaverbeke.nl/blog/codemirror-6-beta.html

[trelljaret]

I use python inside bash a lot. Take this as an example:

#!/usr/bin/env bash

python3 <<EOL
import re

with open("inside.sh", "r") as fp:
    data = fp.read()

data = re.sub(r"enablet=\".*?\" ;", 'enablet="--enable" ;', data, re.IGNORECASE | re.MULTILINE)

with open("inside.sh", "w") as fp:
    fp.write(data)
EOL

git add . ;
git commit -am 'change' ;
git push localenv master ;

Can vscode detect python3 <<EOL until EOL is a python code and syntax highlight and add autocomplete functionality to that?
Is there any plugin for this right now?

[Sayyiditow]

I wish they could highlight SQL syntax in XML files, this is a common practice for us.

[KamasamaK]

For anyone interested in checking the language at the active cursor, regardless of whether it's embedded, in the when clause, I created #107989. It's not as nice as API support, but it's something.

[matu3ba]

Please consider porting treesitter for this. Then the only hard part that remains is gluing the parsers together in a meaningful way.
Also the languages that are glued together must be annotated somewhere in a standard-conforming way.

This has already been implemented for lua-heredoc, but only for full lines and not inline strings or stuff like that (so the context was pretty simple).

[matu3ba]

I use python inside bash a lot. Take this as an example:

#!/usr/bin/env bash

python3 <<EOL
import re

with open("inside.sh", "r") as fp:
    data = fp.read()

data = re.sub(r"enablet=\".*?\" ;", 'enablet="--enable" ;', data, re.IGNORECASE | re.MULTILINE)

with open("inside.sh", "w") as fp:
    fp.write(data)
EOL

git add . ;
git commit -am 'change' ;
git push localenv master ;

Can vscode detect python3 <<EOL until EOL is a python code and syntax highlight and add autocomplete functionality to that?
Is there any plugin for this right now?

you can do this with neovim and treesitter.

[Thom1729]

FYI, I wrote a Sublime package that does this for JavaScript. The user specifies the behavior they want in the package preferences, e.g.:

{
    "defaults": {
        "My Config": {
            "custom_templates": {
                "tags": {
                    "html": "scope:text.html.basic"
                },
                "comments": {
                    "style": "scope:source.css"
                },
                "lookaheads": {
                    "select\b": "scope:source.sql"
                }
            }
        }
    }
}

The package then produces a dynamically generated syntax definition based on the standard JavaScript def, but with the selected features added. In this example, you'd get language-specific highlighting inside each of the following template literals:

html`<strong>Hello, World!</strong>`;
/* style */`color: red`;
`select * from users`;

I imagine that someone could write a similar extension for VSCode.

[vladdeSV]

This is something I would very much like to have.

Similar to one other comment, when dealing with nested languages (eg. SQL in JS) I usually write the code in its own file with highlighting, then copy the code from there.

[xeor]

vetur have done some work around this. There are some magic in https://github.com/vuejs/vetur/blob/master/server/src/embeddedSupport/embeddedSupport.ts and https://github.com/vuejs/vetur/blob/c97734bc89431d9bbc75f129d9593193d80c1800/server/src/services/projectService.ts#L143 is an example-usage for when it does auto-completion.

I think the vetur-way of doing this is a hack tho, so it might be better waiting for this issue.

This should support both

```css
...
```

style, and

<css>...</css>, <style>...</style>

[aeschli]

Thanks for all the input and ideas posted. I'm closing this issue as the issue is difficult to act on.
The issue started overly broad (and ambitious) and has become even broader over time. While we love hearing ideas and use cases, keeping it open raised expectations that we don't know how to fulfil.

I don't see a 'universal' solution to embedded languages that VS Code could implement. I believe that given the language specific syntaxes and mechanisms that are used, every embedding feature has to be solved in the context of the main language of the file.
Generic approaches can solve 80% of the problem but cause frustrations and unsolvable issues for the rest.
So want to encourage people to file feature requests against the language extension.

VS Code offers various APIs for extension developers to add support for embedded languages.
There are mechanisms to

• add syntax highlighting through grammar injection
• add Language features like hovers completions... for embedded through providers
• Language extensions can build on other language support through embedding or forwarding. There are modules providing language smarts such as for css, html, json

[KamasamaK]

• Language extensions can build on other language support through embedding or forwarding. There are modules providing language smarts such as for css, html, json

Fixed the link to the JSON language service.

[godot11]

@aeschli I believe this issue could be significantly narrowed down if we abandon the idea of automatically detecting injected languages. We can let the user itself ask for a specific language instead using comment tags.

For example, in Pycharm, a comment string # language=HTML will automatically cause the string starting beneath the comment to be parsed as HTML code (relevant documentation). I could see something like this in VS Code to inject HTML in a Python script too:

def get_google_location_getter(def_location: str) -> str:
    """ html snippet taken from https://catswhocode.com/html-snippets/ as example """
    loc = handle_loc_str(def_location)
    # language=HTML                    # <-- this causes the next string be handled as HTML language
    loc_html = rf"""
<form action="http://maps.google.com/maps" method="get" target="_blank">
   <label for="saddr">Enter your location</label>
   <input type="text" name="saddr" />
   <input type="hidden" name="daddr" value="{loc}" />
   <input type="submit" value="Get directions" />
</form>
   """
   return loc_html

[aeschli]

@godot11 Please file his suggestion against Python extension. Python language knowledge is needed to properly parse the comment as well as the rf-string.