Memory leak and high CPU usage with actions.RotatePanes with certain pane layouts in wezterm connect unix

[lytedev]

What Operating System(s) are you seeing this problem on?

Linux Wayland

Which Wayland compositor or X11 Window manager(s) are you using?

Hyprland

WezTerm version

wezterm 20230524-062011-1cd340bb

Did you try the latest nightly build to see if the issue is better (or worse!) than your current version?

Yes, and I updated the version box above to show the version of the nightly that I tried

Describe the bug

When I use WezTerm mux'd to a unix domain with a hsplit and a vsplit, the RotatePanes leaks memory and has high CPU usage.

To Reproduce

Run WezTerm like so (with the provided configuration below):

wezterm --config-file /tmp/wezterm-rotate-panes-memory-leak.lua connect unix

Once inside WezTerm, press CTRL-ALT-SHIFT-% and CTRL-ALT-SHIFT-" to get some panes split out. Then press CTRL-ALT-SHIFT-R a couple times (10 if you wanna be sure, but you will need to killall wezterm-gui pretty quick to avoid OOM) and then you can watch memory usage skyrocket.

Note that you can re-attach and everything seems to be perfectly fine from the mux-server's perspective, so the leak seems to be in the gui specifically.

Configuration

-- /tmp/wezterm-rotate-panes-memory-leak.lua
W = require'wezterm'
C = W.config_builder()
C.keys = {
  {
    key = 'r',
    mods = 'CTRL|SHIFT|ALT',
    action = W.action.RotatePanes'Clockwise'
  }
}
return C

Expected Behavior

I wouldn't expect a memory leak nor high CPU usage in this scenario.

Logs

I had to screenshot this because it seizes up pretty badly while the memory is leaking.

Anything else?

Thanks so much for WezTerm! It's a fantastic VTE and I'm a big fan. Thank you for taking the time to look at this bug if you are able!

[wez]

I can't reproduce this as described; both CPU and memory are stable for me with 10 panes and rotating repeatedly. (Admittedly, recomputing and redrawing the panes leads to a brief CPU hit at the time of the rotation, but it's only 15% on my system).

Is there something else happening on your system, or missing from the configuration that you shared that is needed to reproduce this?

How did you determine that wezterm is running out of memory?

[lytedev]

Very interesting! I wonder if shell integration quirks with redrawing in some recursive loop were coming into play? My shell is fish, so I wonder what other variables could come into play here...

I had htop open in another window and watched my system memory climb and climb until the system was unresponsive.

[sublipri]

I can reproduce this on Arch Linux on both X11 and Wayland (using Sway and i3 with zsh). I've been triggering what might be the same issue occasionally when switching between panes on a unix mux workspace that's just two horizontally split panes. The GUI becomes unresponsive and a CPU core is pegged at 99% while memory usage steadily climbs until triggering the OOM reaper. As described above I can kill the GUI process and then re-attach to the mux server without a problem. I usually have 2 wezterm mux workspaces open on different sway workspaces, and both of their GUIs are affected when the issue occurs.

When reproducing with the above config (using main built with the AUR wezterm-git package), I pressed CTRL-ALT-SHIFT-% 4 times, then CTRL-ALT-SHIFT-" 4 times, after which a single CTRL-ALT-SHIFT-R triggered the issue. I've attached debug logs for the GUI and mux server, but can't see anything obvious in them. The rotation KeyEvent that triggers the issue occurs at 13:13:22.269 in the Sway log and 13:33:05.272 for i3. I terminated the processes after a few seconds by sending CTRL-C to the terminal I'd spawned them from.

Once the issue is triggered the GUI logs are spammed with repeated messages:

13:33:05.508  DEBUG  wezterm_gui::termwindow                > Notification(Any { .. })
13:33:05.508  DEBUG  wezterm_gui::termwindow                > Notification(Any { .. })
13:33:05.508  DEBUG  wezterm_gui::termwindow                > Notification(Any { .. })
13:33:05.508  DEBUG  wezterm_gui::termwindow                > Notification(Any { .. })
13:33:05.509  DEBUG  wezterm_gui::termwindow                > Notification(Any { .. })
13:33:05.509  DEBUG  wezterm_gui::termwindow                > Notification(Any { .. })

i3-wezterm-mux-server-log-212519.txt
sway-wezterm-mux-server-log-188390.txt
i3-wezterm-gui-log-212720.txt
sway-wezterm-gui-log-188677.txt

[sublipri]

Seems to be fixed by #4737

Looks like the same root issue where the GUI and mux server get out of sync and cause an endless loop of MuxNotification::PaneFocused notifications

[lytedev]

I'll check it out! Thank you!

[sublipri]

No problem! I should have attached a patch since I had to rebase the PR for it to apply.

4737.patch.txt

Note that according to discussion in the PR it will break the activate-pane and activate-tab CLI commands