Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terminal gets extremely sluggish after a day of use #7710

Closed
lminer opened this issue Sep 23, 2020 · 63 comments · Fixed by #9729
Closed

Terminal gets extremely sluggish after a day of use #7710

lminer opened this issue Sep 23, 2020 · 63 comments · Fixed by #9729
Assignees
Labels
Area-Performance Performance-related issue Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-2 A description (P2) Product-Terminal The new Windows Terminal. Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release.
Milestone

Comments

@lminer
Copy link

lminer commented Sep 23, 2020

I'm finding that if I use Windows Terminal for more than a day, it starts to get extremely sluggish: there is a lag in opening and closing new tabs and a lag in switching between tabs and a lag in typing. This problem immediately resolves itself if I close the application and restart it.

I'm not sure how to give a more detailed bug report. I tried downloading WPR, but it errored out when I tried to make a recording.

My setup is as follows: I'm using the beta version of Terminal, running WSL2 with ubuntu 20.04. I'm on the insider branch of Windows 10 with CUDA support for WSL2 enabled. My laptop is a Razer Blade advanced 2019 4K with an nvidia 2080 mobile graphics card.

@ghost ghost added Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting Needs-Tag-Fix Doesn't match tag requirements labels Sep 23, 2020
@zadjii-msft
Copy link
Member

@lminer (or anyone else seeing this): A dump would be helpful in trying to find the root cause for this issue. You can capture one using Task Manager by right-clicking on WindowsTerminal.exe in the Details tab. Thanks!

@zadjii-msft zadjii-msft added Area-Performance Performance-related issue Issue-Bug It either shouldn't be doing this or needs an investigation. Needs-Repro We can't figure out how to make this happen. Please help find a simplified repro. Priority-2 A description (P2) Product-Terminal The new Windows Terminal. labels Oct 23, 2020
@ghost ghost removed the Needs-Tag-Fix Doesn't match tag requirements label Oct 23, 2020
@zadjii-msft zadjii-msft added this to the Terminal v2.0 milestone Oct 23, 2020
@zadjii-msft zadjii-msft removed the Needs-Triage It's a new issue that the core contributor team needs to triage at the next triage meeting label Oct 23, 2020
@mehgcap
Copy link

mehgcap commented Oct 23, 2020

Just for the record, I'm seeing this as well. I opened a duplicate issue, not realizing this one was already here. I'll try
to capture a dump next time it happens.

@alexandermalfait
Copy link

@lminer (or anyone else seeing this): A dump would be helpful in trying to find the root cause for this issue. You can capture one using Task Manager by right-clicking on WindowsTerminal.exe in the Details tab. Thanks!

Do you mean the "create dump file" option?

@zadjii-msft
Copy link
Member

image

Yep, that one

@lminer
Copy link
Author

lminer commented Oct 23, 2020

How do I send it to you? The files are big

@alexandermalfait
Copy link

@zadjii-msft , you can download a dump file and a video showing the issue from this link: https://we.tl/t-Q9SG7e5z6r

Note that the dump file is 1GB uncompressed, which is a lot of memory usage for a terminal.

In task manager, I could see 700MB of memory usage, dropping to 70MB after restarting the terminal.

@mehgcap
Copy link

mehgcap commented Oct 26, 2020

@zadjii-msft I can only get this to happen reliably when SSH-ed into a few work servers. In case the dump file contains potentially sensitive data, is there an email address where I can send it?

@djsavvy
Copy link

djsavvy commented Oct 29, 2020

@mehgcap see #7981 -- @DHowett provided an e-mail where you can send a dump.

@rebane2001
Copy link

I can confirm it as well, here's a GIF of me typing a sentence in a Windows Terminal vs just WSL
2020-10-30_22-26-22

Usually, when using the Windows Terminal, I have a SSH session through WSL into a tmux session on a server. This server probably prints over a million lines of text a day, so my gut tells me that this data somehow gets stored in a cache/history of some sort that gets laggy once large enough. Of note is also that tmux sends a lot more data than a simple SSH session due to how it works. However, closing tabs does not solve this issue, I need to restart Windows Terminal for it to run fast again.

@alexandermalfait
Copy link

I've just had a (long overdue) 4 day holiday.

Coming back to my desktop session, Windows Terminal was extremely slow.
So the issue seems to "build up" even when WT is not being used at all.

@stedaniels
Copy link

I get this consistently as well, and have for months. I know I shouldn't be leaving terminal windows open over night, but it really shouldn't be a problem either :-) This seems to happen on multiple unrelated Windows 10 installs. E.g. a work AD joined workstation and also my personal Dell PC, and personal homebuilt 7 year old PC.
Do you need any further dumps sending to you?
I'm pretty sure if I just open Windows terminal on a Friday evening, and attempt to use it on a Monday morning (wishful thinking) then the issue would also be present.
Cheers,
Steve

@RoyLi-Pvt
Copy link

I have same problem

@kyuyeonpooh
Copy link

I also have the same problem.

@NeilMacMullen
Copy link

I consistently see this issue (raised as #8640 above) on a machine we use to run some production scripts. Will report further findings here.

@MikeRixWolfe
Copy link

MikeRixWolfe commented Jan 2, 2021

Same issue in both 1.4 release and 1.5 prerelease with WSL1 (Ubuntu 18.04) on my Kaby Lake era desktop build. The issue seems to occur after a few days, closing the terminal (which itself takes up to 10 seconds sometimes) and reopening it fixes the issue. I leave a terminal open constantly (with constant irssi activity).

Edit: because it was brought up below, my terminal usage is also single tab with an ssh+tmux session

@kyuyeonpooh
Copy link

I found that no issue occurs when using terminal with only single tab. I have opened a single-tab terminal for 2 weeks, but no speed problem found while opening new tab or tab switching. Opening multi-tab terminal for a long time may be the cause of this issue.

@Blaok
Copy link

Blaok commented Jan 2, 2021

I found that no issue occurs when using terminal with only single tab. I have opened a single-tab terminal for 2 weeks, but no speed problem found while opening new tab or tab switching. Opening multi-tab terminal for a long time may be the cause of this issue.

My experience is that keeping a single tab only makes the problem shows up slower. After a couple of days it still becomes sluggish when I try to copy from the terminal. My workaround now is to keep a single terminal tab, launch a tmux server in WSL, switch sessions in tmux, and restart the terminal when it's sluggish.

@alexandermalfait
Copy link

No problem! We can all agree babies are vastly more interesting than memory leaks :)

Indeed the link has expired, and I'm searching high and low but can't the original file anywhere.
I restarted my terminal this morning, I'll wait a few days for the issue to pop back up and send you a new dump.

@alexandermalfait
Copy link

PS: is there a way I can send you the link privately?
I guess the dump will contain terminal output, which will probably have sensitive data in it.

@zadjii-msft
Copy link
Member

Sure thing, my email is in my profile ☺️

@desmap
Copy link

desmap commented Mar 19, 2021

Tried Terminal again after a while, wanted to do some comparison/screen recording, all was good and before moving on I went to the kitchen to get me some coffee.

When I came back, my notebook fans were to the max, so opened my task manager and saw this on the "3D" chart:

image

So, I slowly closed first Chrome, then Spotify and eventually, Terminal, take a look:

image

@alexandermalfait
Copy link

@zadjii-msft , I've sent you a new dump over WeTransfer, dit you receive it?

Primary "symptom" I'm seeing is slow tab switching, like a second of delay. After restarting Terminal, tab switching is instant again.

@deepend-tildeclub
Copy link

Sounds like my issue.. Most times I only have one tab open with a SSH session to my server and windows terminal will be using 1.6GB of ram and very slow to respond to clicks and eventually slow to even type in the terminal.
My pc is not slow its 8cores 16threads with 32GB ram, NVME SSD.. So don't understand why it would get sluggish.

@DS992
Copy link

DS992 commented Mar 25, 2021

Experiencing the same issue using WT Preview V 1.7.572.0. I am also a heavy user of tmux (through WSL2).

Clearing the history does nothing. Killing tmux does nothing. It gets slower over time and/or due to the amount of text output in the terminal. For me at least.

Opening a new window (not tab) of WT Preview seems to make it work normally, but the other Windows (which I kept open) is still "sluggish".

So this is not a critical issue, but it's extremely annoying.

@BryceEakin
Copy link

Reported the same issue separately - my mistake. Here's my summary and diagnostic data. Hope it helps get this resolved. Definitely an annoyance, though Windows Terminal has been a huge win in my workflow.

Windows Feedback link for diagnostic data: https://aka.ms/AAbj0dj

@miniksa miniksa self-assigned this Apr 5, 2021
@miniksa
Copy link
Member

miniksa commented Apr 5, 2021

I think I have an idea on this. I need to capture allocation traces for DispatcherTimers as I have a 2GB dump from this state that shows almost 2 million of those allocated and alive in the heap.

@miniksa
Copy link
Member

miniksa commented Apr 5, 2021

ThrottledFunc<> leaks DispatcherTimer instances in two spots:

TerminalControl!<lambda_8050e662c890fd8b88061e60f2bb6336>::operator()+0x6d [E:\BA\3\s\src\cascadia\TerminalControl\ThrottledFunc.h @ 62]

and
TerminalControl!<lambda_922c48371e5965b2d4b2a51a49c8c9c3>::operator()+0x6d [E:\BA\3\s\src\cascadia\TerminalControl\ThrottledFunc.cpp @ 41]

Something about the pattern they're using is making an extra ref/instance that never gets let go and just consumes more and more over time...

@lhecker lhecker assigned lhecker and unassigned miniksa Apr 6, 2021
@ghost ghost added the In-PR This issue has a related PR label Apr 6, 2021
@ghost ghost closed this as completed in #9729 Apr 6, 2021
@ghost ghost removed the In-PR This issue has a related PR label Apr 6, 2021
ghost pushed a commit that referenced this issue Apr 6, 2021
## Summary of the Pull Request

ThrottledFunc previously created a DispatcherTimer whose Tick callback holds a strong reference to the DispatcherTimer itself.
This causes a reference cycle, inadvertently leaking timer instances.

## PR Checklist

* [x] Closes #7710
* [x] I work here

## Detailed Description of the Pull Request / Additional comments

I've initially wanted to remove the `ThrottledFunc<>` optimization, but it turns out that this causes a 3% slowdown. That's definitely not a lot, but enough that we can just keep the optimization for the time being.
I've moved the implementation from the .cpp file into the header regardless since the two implementations are extremely similar and it's easier that way to keep them in line.

## Validation Steps Performed

I've ensured that the scrollbar still updates its length when I add new lines to a newly created tab.
@ghost ghost added the Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release. label Apr 6, 2021
@miniksa miniksa removed the Needs-Repro We can't figure out how to make this happen. Please help find a simplified repro. label Apr 6, 2021
DHowett pushed a commit that referenced this issue Apr 7, 2021
## Summary of the Pull Request

ThrottledFunc previously created a DispatcherTimer whose Tick callback holds a strong reference to the DispatcherTimer itself.
This causes a reference cycle, inadvertently leaking timer instances.

## PR Checklist

* [x] Closes #7710
* [x] I work here

## Detailed Description of the Pull Request / Additional comments

I've initially wanted to remove the `ThrottledFunc<>` optimization, but it turns out that this causes a 3% slowdown. That's definitely not a lot, but enough that we can just keep the optimization for the time being.
I've moved the implementation from the .cpp file into the header regardless since the two implementations are extremely similar and it's easier that way to keep them in line.

## Validation Steps Performed

I've ensured that the scrollbar still updates its length when I add new lines to a newly created tab.

(cherry picked from commit faf372f)
@YAMLcase
Copy link

YAMLcase commented Apr 8, 2021

This sounds very promising! I'm getting about 6 hours of work before I have to wsl --shutdown so would be able to tell almost immediately if there's improvement. Would it be possible to provide a hotfix build?

@zadjii-msft
Copy link
Member

Well, the next release is already scheduled for sometime next week, so probably not? We'd want to selfhost it internally too and make sure we didn't accidentally make this worse 😅

@panicfarm
Copy link

This bug is very serious. I Many programmers need the terminal running for a long time in order not to lose state, but it becomes unusable after several days.

@zadjii-msft
Copy link
Member

@panicfarm then you'll be happy to know that the root cause was found and should be fixed in the next preview build ☺️

@ghost
Copy link

ghost commented Apr 14, 2021

🎉This issue was addressed in #9729, which has now been successfully released as Windows Terminal v1.7.1033.0.:tada:

Handy links:

@NeilMacMullen
Copy link

Fantastic ! Great work team-terminal :-)

@ghost
Copy link

ghost commented Apr 14, 2021

🎉This issue was addressed in #9729, which has now been successfully released as Windows Terminal Preview v1.8.1032.0.:tada:

Handy links:

@mehgcap
Copy link

mehgcap commented Apr 15, 2021

So far, so good, at least on an SSH connection I left open to a work server for 24 hours. I did notice that things quickly got sluggish in the settings window, of all places, though I have no idea if the cause is the same. Still, at least the actual terminal part seems to be working much better. Thank you to those who put in the work to solve this one.

@YAMLcase
Copy link

I noticed a marked improvement, but still find my terminal getting into sluggish mode. I most frequenly notice it when lots of lines displaying on a remote ssh session (i.e., cat big logfile). the latest WSLTTY doesn't get sluggish with the same test.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Performance Performance-related issue Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-2 A description (P2) Product-Terminal The new Windows Terminal. Resolution-Fix-Committed Fix is checked in, but it might be 3-4 weeks until a release.
Projects
None yet
Development

Successfully merging a pull request may close this issue.