[editors/code] add markdown syntax highlighting to doc comments

[AndreasBackx]

This is a continuation of microsoft/vscode#169956 and dustypomerleau/rust-syntax#37 (that repo is no longer maintained: dustypomerleau/rust-syntax#39 (comment)). The VS Code team seemed to prefer this being inside of an extension.

This adds Markdown highlighting to doc comment lines and blocks. Currently it is thus regarded both as a comment and as Markdown which leads to normally foreground text being in the colour of the comment and the rest highlighted like Markdown or its own embedded languages in code blocks.

Block comments are supported, but currently not when there is a * at the start of the line:

I'm not entirely sure if I can easily fix this, I'd need to find a way to make the content ignore the *. Though I'm unsure if it's important as there are conventions against using block comments and using them without * does work. All of this TextMate grammar is so hard to find documentation on that honestly I'd just not want to support this considering the effort.

Let me know what everyone thinks of this being in rust-analyzer. I've personally found it hard to write large amounts of Rust documentation due to the lack of Markdown syntax highlighting.

Also, thank you @adenine-dev as well for making this available in the interim and your enthousiasm. Wanted to get this PR out sooner, but life gets in the way.

[Veykril]

Seems fine to add, though ideally we'd replaces this with semantic highlighting once we parse doc comments as markdown syntax trees
@bors r+

[bors]

📌 Commit 09f6247 has been approved by Veykril

It is now in the queue for this repository.

[bors]

⌛ Testing commit 09f6247 with merge 8ad70e7...

[bors]

☀️ Test successful - checks-actions
Approved by: Veykril
Pushing 8ad70e7 to master...

[Veykril]

So I just realized, but this does not work for me 🤔 Is there something specific one has to do in VSC to make injected grammars work?

[AndreasBackx]

So I just realized, but this does not work for me 🤔 Is there something specific one has to do in VSC to make injected grammars work?

I'll have a look this week why it doesn't seem to work. I'm not at home for the foreseeable time and so need to find some time to setup rust-analyzer. It might have to do with the fact that I was testing this against the built-in Rust extension as well and it was quite fragile. Perhaps I made a mistake when making the PR or there is something different in production, I think it's the former. I'll try to get back later this week.

[AndreasBackx]

So I am doing some debugging and am facing a weird bug with VS Code. When testing the extension as is right now, it does work but it weirdly starts of looking like this:

Then as soon as I do one of the following actions:

• Take said screenshot
• Change colour theme (and reverting to the default colour theme does not revert the behaviour)
• Changing out of the editor and back in

I haven't thoroughly tested it, but any of them seem to then "fix" the issue:

So it is doing some loading somewhere that I don't know enough about. My gut tells me this is a VS Code TextMate grammar injection bug because of how we're overriding the begin, end, and while TextMate stuff and, iirc, I saw this behaviour in the past because of how those previously did not account for what's "inside" the comments. For now, I am going to ignore it so I can move on with verifying why it works in testing but does not seem to work in the release. I am assuming it has something to do with how the VS Code extension gets rolled out.

[AndreasBackx]

Thanks @Veykril for reporting the issue as well, it should be fixed now in #15032.

[AndreasBackx]

Also following up from the previous bug. I did not notice this behaviour when using the vsix in testing. It might be a bug only visible during testing. Let's see if users face the issue, if so, we can look more into it and possibly report it on the VS Code issue tracker.