Serious Performance Issues with Syntax Highlights Enabled

[fitdev]

For reference, I am using latest Codist 5.11.0.6426 on VS 2019 16.11 Preview 1. However this is not specific to this version of Codist or VS - I have had similar issues for a long time. It's just that now it has gotten particularly bad (it seems to "not like" certain code for whatever reason), so I decided to investigate a bit.

What I found is that generally there is a very noticeable lag when typing when Syntax Highlight module is enabled (I tested various combinations of which modules in Codist are enabled and the offender seems to be Syntax Highlight. Basically when it is on, there is like half-a-second to a full second lag between typing characters and them appearing (of course it varies depending on the document, its size, etc., but on average it's that).

Comment Tagger seems not to be the issue (since there is no difference between whether it is enabled or not).

I don't know the details of how Syntax Highlight is implemented in Codist, but my suggestion would be to postpone all work in this module until typing has stopped (i.e. > 1 second has passed). I am sure VS must inform somehow whether a user is actively typing or not. Because triggering a syntax highlight on every keystroke is very performance intensive. Alternatively maybe all this work can be shifted off the UI thread?

This is really critical, and for the moment I am forced to disable Syntax Highlight as it is preventing me from getting work done.

Note: it works ok in very small solutions, but any sizable solution with 10+ projects and a few hundred files will already lead to serious slowdowns. So, perhaps you can find a way to postpone triggering syntax highlight until typing is finished?

[wmjordan]

Thank you for reporting.

I will check to see what's blocking the user interface.

[wmjordan]

I debugged and found that the syntax highlight was performed on the main thread!
So strange!

Please try the new beta.

[fitdev]

Thank you for looking into this and providing a new beta so quickly.

Strangely, I never got that VS warning popup about Codist slowing things down.

Unfortunately the new beta has even more issues. Basically performance stayed the same (very slow typing when Syntax Highlight is on). But now also the syntax highlights are almost always wrong:

Here you can see multiple issues with inconsistent colors and font styles being applied to different parts of the same symbol: Equals, GetHashcode, int, operator. And what's more it changes with every keystroke, indicating that it is reapplying the style very often, which is what in all likelihood causing the performance issue.

I understand this may be a major task, but perhaps you can rethink how this feature is implemented in an effort to minimize the frequency of style recalculations, and limit them to a narrow scope (like only a few lines that have changed in the last couple of seconds). Then, once the user stopped typing, you can "recalculate" the whole document, which may be needed since new braces / types / etc might have been introduced that could significantly change how the document should be highlighted.

[fitdev]

Config.json.zip

Here's my Config.json in case you need it.

And these are VS C# Advanced Settings.

Also I am using DotNet 6 Preview 4 if it matters (but I don't think it should matter).

[wmjordan]

I met with the slow-down when I opened the project Roslyn in VS 2019.10.

If you disabled Codist completely, would you be able to type faster in your huge solution?

I tested the beta with the project Roslyn.
It was a bit faster and I did not encounter the weird highlight results.

[fitdev]

If you disabled Codist completely, would you be able to type faster in your huge solution?

Absolutely! Much, much faster - basically then there is no slowdown and everything is smooth.

But I do not need to disable entire Codist - only the Syntax Highlight module. If I disable just the syntax highlight module and then reopen the document, I can type fast. No need to disable the whole Codist.

[wmjordan]

OMG, as I dug into this issue.... I found it a big task...

[fitdev]

Hopefully it will worth it in the end!
Otherwise this feature (and I Love it) is not usable on medium/large solutions.

[wmjordan]

There's a new beta but it does not seem to fully solve this issue.
Please give it a try anyway.

[fitdev]

Thank you for the new beta. Still seems very laggy compared to when it is turned off. But the inconsistent rendering issue is now gone and everything is highlighted properly and consistently.

[wmjordan]

I might have to read the source code of Roslyn and develop a similar approach like them to make the syntax highlight running in another thread.

[fitdev]

I see. Well, I hope you will succeed with this! You have the best syntax highlighter, and it will be really bad if the feature is unusable because of performance issues.

Also just wanted to mention that some features of syntax highlighting like user vs. non-user code are especially useful in large solutions (where there are more serious performance impacts): simply because in a large codebase you may not always remember which ones are framework methods/types and which ones are yours.

[wmjordan]

Hey, please download the new beta and give it a try. I am trying to make the highlight in asynchronous threads. It's performance should be much better now. Yet it is supposed to be buggy.

However, I at least did not notice the terrible delay when typing into a huge solution (e.g. the Roslyn projects).

[fitdev]

Thank you for working on this! I will give it a try and let you know soon. From the quick test I have done already - it does seem to be a lot faster now. I will see how it goes.

[wmjordan]

Thank you for your feedback.
Still quite a few bugs are yet to be fixed.
Please download a new beta which has fixed some of them.

[fitdev]

Performance seems to be a lot better. It is now usable again! Thanks for your great work!

I have not encountered any major bugs related to syntax highlighting.

But I did get VS crash when renaming a symbol (Property name in an interface):

Application: devenv.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.ArgumentOutOfRangeException
   at Microsoft.CodeAnalysis.SyntaxNode.FindNode(Microsoft.CodeAnalysis.Text.TextSpan, Boolean, Boolean)
   at Codist.SyntaxModel.GetNode(Int32, Boolean, Boolean)
   at Codist.SemanticContext.GetContainingNodes(Int32, Boolean, Boolean)
   at Codist.NaviBar.CSharpBar+<UpdateModelAndGetContainingNodesAsync>d__18.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
   at Codist.NaviBar.CSharpBar+<<Update>g__Update|17_0>d.MoveNext()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
   at Codist.NaviBar.CSharpBar+<Update>d__17.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.<ThrowAsync>b__6_0(System.Object)
   at Microsoft.VisualStudio.Threading.JoinableTaskFactory+SingleExecuteProtector.TryExecute()
   at Microsoft.VisualStudio.Threading.JoinableTaskFactory+SingleExecuteProtector+<>c.<.cctor>b__20_0(System.Object)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
   at System.Windows.Threading.DispatcherOperation.InvokeImpl()
   at System.Windows.Threading.DispatcherOperation.InvokeInSecurityContext(System.Object)
   at MS.Internal.CulturePreservingExecutionContext.CallbackWrapper(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at MS.Internal.CulturePreservingExecutionContext.Run(MS.Internal.CulturePreservingExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Windows.Threading.DispatcherOperation.Invoke()
   at System.Windows.Threading.Dispatcher.ProcessQueue()
   at System.Windows.Threading.Dispatcher.WndProcHook(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
   at MS.Win32.HwndWrapper.WndProc(IntPtr, Int32, IntPtr, IntPtr, Boolean ByRef)
   at MS.Win32.HwndSubclass.DispatcherCallbackOperation(System.Object)
   at System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate, System.Object, Int32)
   at System.Windows.Threading.ExceptionWrapper.TryCatchWhen(System.Object, System.Delegate, System.Object, Int32, System.Delegate)
   at System.Windows.Threading.Dispatcher.LegacyInvokeImpl(System.Windows.Threading.DispatcherPriority, System.TimeSpan, System.Delegate, System.Object, Int32)
   at MS.Win32.HwndSubclass.SubclassWndProc(IntPtr, Int32, IntPtr, IntPtr)

[wmjordan]

Oh, I am sorry for the crash.

I planned to rewrite the underlying stuff for NaviBar, SmartBar, etc. after that the updated highlight module becomes stable.
Afterward, Codist should be running even faster and save some energy.

[wmjordan]

How is your working experience with the beta version?
If no problem, I am planning to release this version and get on with optimizations on NaviBar, SmartBar, etc. afterwards.

[fitdev]

It seems to be pretty good! But I have not done very extensive testing yet.
BTW, any plans to release 64-bit version for VS 2022 (the first preview is already out)?

[wmjordan]

No preview version support yet.
I am toooooo busy to have 3 VSes on my computer.
If it is released, I will work on it.

[wmjordan]

While I debugged Codist, I was surprised to find that some async functions in event handlers were running on the Main Thread, which obviously would cause the slowdown--as you have experienced!

So, the slowdown might not be caused by the highlighter, but other components.
That was also the reason that I was planning to rewrite the other components.

[fitdev]

Thank you for pointing this out. But I have Scrollbar Markers module disabled. I only have Super Quick Info, Syntax Highlight, and NaviBar modules enabled.

[wmjordan]

The NaviBar could also potentially slow down VS--I guess.
Please try a new beta version.

[fitdev]

The new beta seems to be ok. Though if possible, in the future, you might consider changing the implementation a bit so that it is smarter about the scope of file that needs styles to be recalculated and updated. Too often I see VS revert to its default highlighting momentarily, as I type, and then Codist replaces that original highlighting with its highlighting - and it happens for the whole document, instead of affecting only the nearby lines where I type. But at least it is more performant now.

[wmjordan]

Please try the new beta.
This one should better prevent VS from reverting to its default highlight and perform a little bit better.

[fitdev]

Thank you for the new beta. Does it address the memory leak too, or just the highlighting part?

[wmjordan]

Just the highlight part.

[wmjordan]

The latest beta had addressed both the memory leak and the highlighting part.
Also, it fixed a problem in the C# Immediate window (caused by previous beta builds) too.

[fitdev]

Thanks so much for your work! I will try it ASAP. However turns out at the moment I am having issues with VS itself - at least VS 2019 16.11 Preview 2 (latest) - even with Codist completely disabled, I very quickly get to the point of it running out of memory and it effectively freezing. In other words, I cannot properly test Codist's performance since VS itself may be the culprit.

I will switch to VS 2022 Preview for the moment since there memory issues are not as apparent. I did manage to get Codist to work with it too (only Syntax Highlight and NaviBar though) Super QuickInfo does not work. So, perhaps if you can start supporting VS 2022, it will be great.

[wmjordan]

Have you experience memory leak with the latest Codist on VS 2019?
I suspected that it was not fully fixed. How is the situation on your side?

[fitdev]

It's still hard to say. I am keeping an eye on this. But as I told you a couple of days before the problem with VS 2019 for me is that it seems to run out of memory even with Codist fully disabled. So in my case at least it was not (only) a Codist-related issue. Right now I have Codist enabled in VS 2019 and will see how it goes.

If you are curious, here is the screenshot of the heap map that I took when VS was running out memory with Codist:

And without Codist:

Not much difference.

[fitdev]

But as far as the performance of the highlighting is concerned - it seems to be a lot better now than what it was! So, good work there!

[fitdev]

Actually you are probably right about memory still leaking. After VS 2019 was close to run out of memory, I took another heap snapshot:

I suspect that EventHandler type and perhaps some other types have something to do with Codist, which was enabled for the session - the NaviBar, Syntax Highligher, and Super QuickInfo.

[wmjordan]

Thank you very much for helping address this issue.
The leak has something to do with the recent updates.

I will check why so many ParserTasks are keep there.

[fitdev]

Although this is from VS 2022 (I had to switch to it, because VS 2019 is hardly usable at this point for any period of time), there is a similar kind of thing:

In particular:

EventHandler - not 100% sure it is Codist-related though.
Codist.Taggers...
And perhaps a few other items - you would know better of course which might be related to Codist.

And sorted by size:

And filtered to Codist only:

[wmjordan]

Please download the new beta which has addressed the leak.

[fitdev]

Thank you for the beta. I have installed it and will let you know how it goes in a few days.

[wmjordan]

The leak has not been fully fixed. Working on it...

[wmjordan]

Please download and try the new beta.

I've been working with the latest beta for an afternoon. So far, no leak has been observed via WinDbg.

[fitdev]

Thank you. Downloaded it. Will let you know if everything is ok in a few days.

[wmjordan]

This beta should be a little better.
However, I worked a whole morning with the new version and found that there was still some leak if we kept editing a document without closing it.

I will upload a new beta later.

[fitdev]

Thank you for your efforts! Hope you can start supporting VS 2022 soon, as I am forced to use this version due to VS 2019's memory issues (unrelated to Codist).

[wmjordan]

There's a new beta. Please give it a try.

[fitdev]

Thank you! Sorry I did not provide much feedback. But I have not been doing extensive coding in the last few days, and was using VS 2022 more often. Now that VS 2022 Preview 2 is out, hopefully you can make an official version for it too.

I will give the new beta a try and let you know in a couple of days.

[fitdev]

Sorry for long delay. But at this point I cannot give Codist an accurate assessment, because right now for me VS 2019 is not usable (runs out of memory and becomes unresponsive very quickly), and VS 2022 you do not officially support (but it has its own performance issues).

I think, however, you have already done a good job, so perhaps this issue can be closed. And when I sort out things with VS, if there will still be Codist-related slowdowns, I will open a new issue.

I hope you can release official version for VS 2022 soon (that's what I am using now because of VS 2019's memory issues). They already released Preview 2, and encourage people who use latest DotNet SDKs (like I do) to switch. I think supporting it should be straight forward, as I already managed to get Codist half-working there.

[wmjordan]

Thank you for your reply. I occasionally checked the memory usage of VS 2019 with WinDbg and no longer found serious leak, after days' working with the last beta. I will release this version soon.

The crash of the NaviBar may because VS 2022 has removed a certain DLL files that exist in prior versions. According to the extension migration guide, a separate project is required to build Codist for VS 2022. My schedule is tight at this moment and I am sorry that I can not afford the time on the unstable VS 2022 or promise anything.