C/C++ extension does not work with Unicode identifiers

[BErasmus]

I use Unicode identifiers everywhere in my C++ code. It is well supported in Visual Studio. There are some bugs with syntax highlighting for example, but nothing major.

Unfortunately with Visual Studio Code it basically does not work at all. It looks like the problem is caused by the all too common mistake of some code counting in characters interfacing with other code that counts in bytes. Some examples below.

When you bring up the function arguments of a function with a Unicode argument you get a strange "error-type" message.

Note also when the cursor is on a1, the other a1 is also highlighted. But when the cursor is on a2, the Ç character is highlighted.
And note how the color of ABC in struct ABC {}; is different from the color of ḀʙÇ in struct ḀʙÇ {};

Tested on Windows 10, VS Code 1.23.0, C/C++ extension 0.17.3.

[sean-mcmanus]

UTF-8 characters > 1 byte should work in identifiers now with our latest Insiders release: https://github.com/Microsoft/vscode-cpptools/releases . If you're using VS Code Insiders or have http.proxySupport not set to "off", you might not get the 0.21.0-insiders2 update automatically with C_Cpp.updateChannel set to "Insiders".

[BErasmus]

Glad to see that UTF-8 is now much better supported. I noticed that some syntax highlighting problems remain, eg identifiers shown in the wrong color:

[bobbrow]

Those colorization bugs would be due to the Atom grammar that is currently used by VS Code to color the file. #230 is on our backlog and should cover this case.

[sean-mcmanus]

Or after VS Code supports tree-sitter: microsoft/vscode#50140 .