Grammars shouldn't be able to freeze VS Code

[alexr00]

I often get reports of hangs that are caused by a grammar. It would be better if we didn't force a restart of VS Code every time this happens.

From #89846

instead of the current "not responding, would you like to restart" message, have something like "The is not responding. Would you like to view this file in plain text mode instead? "

[alexdima]

This is a good suggestion. Here is the entire picture:

• the message "not responding, would you like to restart" is a generic message shown by the main process when a renderer process stops responding. That can be for any number of reasons, one of which is runaway regular expressions.
• tokenization is currently implemented on the main thread / UI thread in the renderers and it works with a custom regex engine, oniguruma. Oniguruma is written in C++ and we end up calling into that library. In certain cases, with certain inputs and regular expressions, it takes oniguruma a very very long time (minutes, or hours) to return. This situation is called runaway regular expresions / catastrophic backtracking.
• unfortunately, oniguruma does not provide currently a mechanism to terminate the matching after X amount of time or after X amount of backtrackings, so we cannot call it differently.
• the only way I can think of to workaround this is to load up oniguruma on a separate thread (not the UI thread). But there are difficulties with this because tokens are needed synchronously in certain cases while typing. This is discussed/planned in Tokenization overhaul #77140

I share the frustration and I don't think a grammar should end up hanging VS Code, but I suggest we continue tracking #77140

Also, in any case, regardless of vs code hanging or not, the regex must be rewritten. This is ultimately a mistake in the grammar. Regexes which lead to catastrophic backtracking are poorly written regexes and must be rewritten.