Use semantic highlighting

[edvgui]

The language server protocol allows a server to provide information about semantic tokens, which can, from the editor side, be mapped to some nice coloring. It would for example allow us to color the entity names, plugin names, namespaces path, variable, etc.

I started exploring this, and I have a really short PoC, but it misses some elements to work correctly.

1. The server should communicate that it can provide semantic tokens, and give the list of tokens it might return

--- a/server/src/inmantals/server.py
+++ b/server/src/inmantals/server.py
@@ -122,6 +122,14 @@ class InmantaLSHandler(JsonRpcHandler):
                     # the language server does not report work done progress for workspace symbol requests
                     "workDoneProgress": False,
                 },
+                "semanticTokensProvider": {
+                    "legend": {
+                        "tokenTypes": ["type"],
+                        "tokenModifiers": [],
+                    },
+                    "range": False,
+                    "full": True,
+                }
             }
         }
 

2. The server will then receive textDocument/semanticTokens/full requests, which it can answer to with a list containing the the position of tokens, their length, and the token in question (its index in the list provided in the capabilities config returned by the server)

    async def textDocument_semanticTokens_full(self, textDocument):
        """
        https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#scope-inspector
        https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide
        https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_semanticTokens
        """
        uri = textDocument["uri"]

        url = os.path.realpath(uri.replace("file://", ""))

        # line, start char, length, tokenType

        logger.info("Providing semantic tokens for file %s", url)

        # This says that at the first line of the file, at position zero and for a length
        # of 6 characters, the token present there is a type
        return {"data": [1, 0, 6, 0]}

3. When running the extension, and using the token inspector, you can then see that the information is passed through and accessible by vscode. We just need to setup some coloring for it.

Screenshare.-.2023-03-13.8_03_42.AM.mp4

Here is a list of things that should still be figured out:

• How to extract the semantic tokens information from the compiler, this should ideally be done after parsing, and before the compiler runs completely (we want semantic coloring even if the compile fails)
• How to setup the coloring in the extension

Some useful references:

• https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#scope-inspector
• https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide
• https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_semanticTokens