A barrage of network-congestion prompts

[stdedos]

YES, I get it, something is up with the network 😛

Please staph now!

"Xpra-Python3-x86_64_4.2-r28877\xpra_cmd" attach ssh://user@ip/4 --ssh="plink -ssh -agent" --modal-windows=no --title="@title@ on @@/@server-display@" --headerbar=off --opengl=no --bandwidth-limit=6Mbps

XPRA_CUSTOM_TITLE_BAR=0
XPRA_EXECUTABLE=Xpra-Python3-x86_64_4.2-r28877
XPRA_REPAINT_ALL=1
XPRA_SCROLL_ENCODING=0

2021-04-14 12:08:46,537 Xpra GTK3 client version 4.2 64-bit
2021-04-14 12:08:46,540  running on Microsoft Windows 10
2021-04-14 12:08:48,502 GStreamer version 1.18.4 for Python 3.8.9 64-bit
2021-04-14 12:08:48,979 created named pipe 'Xpra\18360'
2021-04-14 12:08:49,464 keyboard layout code 0x409
2021-04-14 12:08:49,465 identified as 'United States - English' : us
2021-04-14 12:08:50,251  keyboard settings: layout=us
2021-04-14 12:08:50,262  desktop size is 4160x1440 with 1 screen:
2021-04-14 12:08:50,264   Default (1100x381 mm - DPI: 96x96) workarea: 4160x1400
2021-04-14 12:08:50,272     Generic PnP Monitor 1600x900 at 0x534 (309x174 mm - DPI: 132x131) workarea: 1600x860 at 0x534
2021-04-14 12:08:50,272     C32JG5x 2560x1440 at 1600x0 (697x392 mm - DPI: 93x93) workarea: 2560x1400 at 1600x0
2021-04-14 12:08:55,240 enabled remote logging
2021-04-14 12:08:55,243 Xpra GTK3 X11 server version 4.2-r28877 64-bit
2021-04-14 12:08:55,243  running on Linux Ubuntu 20.04 focal
2021-04-14 12:08:55,254  (press Control-C to detach)

Nothing exceptionally wrong in the server log, with the exception of 7-8k send_delayed_regions: using full window update, merged region covers almost the whole window messages

[totaam]

Didn't ignore avoid any further warnings?
Or do you mean that the multiple warnings occurred before that?

[stdedos]

When they start, they just won't stop (see the taskbar, and imagine that as a video).
Also, the empty space on the top-right, is yet another message as the one below it (not visible in this screenshot), flashing.

Interacting with any of them is impossible, closing them via explorer.exe is impossible, xpra is at 18.5% CPU, explorer.exe at 35%!!

The only solution is to kill the client, and reconnect. (xpra upgrade for the xdg-open didn't help)

[totaam]

Ah. Gotcha.

So there are many pieces to this puzzle:

• why you're getting congestion events is one: -d bandwidth on the server should tell us
• the notification should be unique (the fix above deals with that - tested standalone - new MS Windows build uploaded with this fix)
• why having so many notification windows DoSed your desktop - not going to worry too much about this, let's just not do that from now on

xpra upgrade for the xdg-open didn't help

I don't understand this bit.

[stdedos]

• why you're getting congestion events is one: -d bandwidth on the server should tell us

Pending replication.
As soon as you responded, it started "not replicating" 😛

• the notification should be unique (the fix above deals with that - tested standalone - new MS Windows build uploaded with this fix)

Make sure that there is some minimum delay between "modifying" the window, if that would make it unable to be interacted with (e.g. 500ms is a good enough tradeoff on updating vs able-to-be interacted with).
I am seeing something in the ballpark of 6 windows per less-than a second (as you can see in the screenshot above); I don't know if your 6 window modifications/sec will allow or inhibit interaction.

• why having so many notification windows DoSed your desktop - not going to worry too much about this, let's just not do that from now on

No idea what that is 😀

xpra upgrade for the xdg-open didn't help

I don't understand this bit.

I was thinking if this #3089 (comment) would somehow affect. I did see the problem again after upgrading to fix that.

[totaam]

Make sure that there is some minimum delay between "modifying" the window...

Maybe this isn't working:

CONGESTION_REPEAT_DELAY = envint("XPRA_CONGESTION_REPEAT_DELAY", 60)

This should ensure that you don't get more than one notification per minute..
And you also need more than CONGESTION_WARNING_EVENT_COUNT (defaults to 10) signals.

Adding -d notify,bandwidth should tell us what's going on there. (useful for both client and server)

[stdedos]

No, definitely the client did not honor that from where I was sitting (the way I understand it at least)

xpra/xpra/server/source/windows_mixin.py

Line 27 in 9e5f3ce

CONGESTION_REPEAT_DELAY = envint("XPRA_CONGESTION_REPEAT_DELAY", 60)

[totaam]

Found the problem: f8b2ef3 uses idle_add to fix some potential threading issues first spotted on macos (#3049).
The problem is that idle_add runs again, immediately and forever! No wonder it was killing your UI: it was creating windows as fast as it could!

I am so confident that this fix is correct that I am closing this ticket already. (the latest beta builds uploaded have this fix)

[stdedos]

Successfully disappeared:

"Xpra-Python3-x86_64_4.2-r28905\xpra_cmd" attach ssh://user@ip/2 --ssh="plink -ssh -agent" -d notify,bandwidth -d clipboard --modal-windows=no --title="@title@ on @@/@server-display@" --headerbar=off --opengl=no --bandwidth-limit=6Mbps

XPRA_EXECUTABLE=Xpra-Python3-x86_64_4.2-r28905

2021-04-16 12:28:26,458 Xpra GTK3 client version 4.2 64-bit
2021-04-16 12:28:26,465  running on Microsoft Windows 10
2021-04-16 12:28:31,868 GStreamer version 1.18.4 for Python 3.8.9 64-bit
2021-04-16 12:28:32,688 created named pipe 'Xpra\2104'
2021-04-16 12:28:33,284 keyboard layout code 0x409
2021-04-16 12:28:33,301 identified as 'United States - English' : us
2021-04-16 12:28:33,777 make_clipboardmenuitem()
2021-04-16 12:28:34,452  keyboard settings: layout=us
2021-04-16 12:28:34,477  desktop size is 4160x1440 with 1 screen:
2021-04-16 12:28:34,482   Default (1100x381 mm - DPI: 96x96) workarea: 4160x1400
2021-04-16 12:28:34,483     Generic PnP Monitor 1600x900 at 0x534 (309x174 mm - DPI: 132x131) workarea: 1600x860 at 0x534
2021-04-16 12:28:34,484     C32JG5x 2560x1440 at 1600x0 (697x392 mm - DPI: 93x93) workarea: 2560x1400 at 1600x0
2021-04-16 12:28:38,794 DISCARD_TARGETS=re.compile('^NeXT'), re.compile('^com\\.apple\\.'), re.compile('^CorePasteboardFlavorType'), re.compile('^dyn\\.'), re.compile('^resource-transfer-format'), re.compile('^x-special/')
2021-04-16 12:28:38,800 DISCARD_EXTRA_TARGETS=re.compile('^SAVE_TARGETS$'), re.compile('^COMPOUND_TEXT'), re.compile('GTK_TEXT_BUFFER_CONTENTS')
2021-04-16 12:28:38,801 server clipboard: supported=True, direction=both
2021-04-16 12:28:38,803 client clipboard: supported=True, direction=both
2021-04-16 12:28:38,804 parse_clipboard_caps() clipboard enabled=True
2021-04-16 12:28:38,806 enabled remote logging
2021-04-16 12:28:38,808 Xpra GTK3 X11 server version 4.2-r28886 64-bit
2021-04-16 12:28:38,815  running on Linux Ubuntu 20.04 focal
2021-04-16 12:28:38,817 process_ui_capabilities() clipboard_enabled=True

....

2021-04-16 14:22:24,726 CloseClipboard()=1
2021-04-16 14:22:24,730 clipboard event: CLIPBOARDUPDATE, current owner: our window (hwnd=0x140a0a)
2021-04-16 14:22:24,756 remove_block: CLIPBOARD
2021-04-16 14:24:21,899 process_clipboard_packet: clipboard-token, helper=ClipboardProtocolHelperCore
2021-04-16 14:24:21,910 ignoring token for clipboard 'PRIMARY' (no proxy)
2021-04-16 14:25:47,981 process_clipboard_packet: clipboard-token, helper=ClipboardProtocolHelperCore
2021-04-16 14:25:47,987 process clipboard token selection=CLIPBOARD, local clipboard name=CLIPBOARD, proxy=Win32ClipboardProxy
2021-04-16 14:25:47,989 wire selection to raw, encoding=bytes, type=UTF8_STRING, format=8, len(data)=11
2021-04-16 14:25:47,991 got token, selection=CLIPBOARD, targets=(b'UTF8_STRING', b'STRING'), target data={'UTF8_STRING': ('UTF8_STRING', 8, b'N\n')}, claim=True, can-receive=True
2021-04-16 14:25:47,993 _filter_targets(UTF8_STRING, STRING)=('UTF8_STRING', 'STRING')
2021-04-16 14:25:47,994 _filter_targets(UTF8_STRING, STRING)=('UTF8_STRING', 'STRING')
2021-04-16 14:25:47,996 got_contents: tell OS we have UTF8_STRING, STRING
2021-04-16 14:25:48,002 we got a byte string: b'N\n'
2021-04-16 14:25:48,003 MultiByteToWideChar wlen=12
2021-04-16 14:25:48,005 GlobalAlloc buf=0x18ae0088
2021-04-16 14:25:48,010 with_clipboard_lock(1313290, <function Win32ClipboardProxy.set_clipboard_text.<locals>.set_clipboard_data at 0x00000000071615e0>, <function Win32ClipboardProxy.set_clipboard_text.<locals>.set_clipboard_error at 0x0000000007161ca0>, 5, 10)
2021-04-16 14:25:48,014 OpenClipboard(0x140a0a)=1
2021-04-16 14:25:48,027 clipboard event: DESTROYCLIPBOARD, current owner: our window (hwnd=0x140a0a)
2021-04-16 14:25:48,028 EmptyClipboard()=1
2021-04-16 14:25:48,040 SetClipboardData(CF_UNICODETEXT, 12 chars)=414056584
2021-04-16 14:25:48,041 <function Win32ClipboardProxy.set_clipboard_text.<locals>.set_clipboard_data at 0x00000000071615e0>()=True
2021-04-16 14:25:48,042 CloseClipboard()=1
2021-04-16 14:25:48,044 clipboard event: CLIPBOARDUPDATE, current owner: our window (hwnd=0x140a0a)
2021-04-16 14:25:48,052 remove_block: CLIPBOARD

---- NOTIFICATION ---

2021-04-16 14:29:04,533 server is not responding, drawing spinners over the windows
2021-04-16 14:29:04,849 server is OK again

I don't see anything useful
server-display.zip
(sorry for the archive-in-archive; I didn't know Github was so picky of extensions 😕)

I think the buttons to interact with have disappeared too, however 😕
(Didn't have time for a screenshot; it only happened once)

[totaam]

I think the buttons to interact with have disappeared too, however confused

That means the popup was being shown again - fixed in 3d5ac37.

From your server log, the key congestion events are:

10:43:09,373 record_congestion_event(late-ack for sequence   9637: late by 10529ms, target latency=4443 ((46, 42, 4255, 100)), 24816, 8289) bandwidth_warnings=True, elapsed time=672266
...
10:43:09,428 record_congestion_event: 10 events in the last 10 seconds (warnings after 10)

Then again at 14:29:28,261 record_congestion_event: 11 events in the last 10 seconds (warnings after 10)

Both times something was really wacky with the network: the screen update packets should not take 10 seconds to do the roundtrip!

[stdedos]

Will more frequent server ping-pongs portray the problem better?
Or you are "sure enough" that this is simply network-related?

For the record, the situation is much more stable and usable on Focal.
So far, everything I've done has happened over WiFi, and any kind of window can be used in either the HD or the WQHD display with no penalty at all.