shadow desktop ceases to update when using --desktop-scaling < 100%

[Lanchon]

Describe the bug
shadow desktop ceases to update a couple of seconds after starting when using --desktop-scaling < 100%. this happens regardless of whether OpenGL is used. mouse actions in the client to server direction keep on working. no text is displayed in the client console when the image ceases to update.

To Reproduce
Steps to reproduce the behavior:

1. server command: none
2. client command:
   • xpra shadow ssh:XXX:0 --desktop-scaling=70%
   • xpra shadow ssh:XXX:0 --desktop-scaling=70% --opengl=off
3. specific action to trigger the bug: wait 2 seconds

System Information (please complete the following information):

• Server OS: Linux Mint 22 Wilma base: Ubuntu 24.04 noble
• Client OS: Linux Mint 22 Wilma base: Ubuntu 24.04 noble
• Xpra Server Version: xpra v6.1.1-r0
• Xpra Client Version: xpra v6.1.1-r0

Additional context
on both sides:
Desktop: Cinnamon v: 6.2.9 tk: GTK v: 3.24.41 wm: Muffin vt: 7 dm: LightDM v: 1.30.0
standard X11 session

[totaam]

That's odd, is there anything in your client output at all?
Your server log perhaps?

I did hit some major painting issues with opengl, but none without.
The opengl issues are now fixed (and have already been backported to 6.1.x and will be included in the upcoming v6.1.2 release)
I've just tried it here with Fedora 40, xpra from git master and everything worked as expected - attaching locally with --no-mmap.

[Lanchon]

i downgraded only the client, then tested several client versions against 6.1.1-r0:

(freezes = stops updating the client windows a couple of seconds after launching)
(works = no freezing)

version	opengl off	opengl default
5.0.9-r0-1	works (bilinear interpolation)	works (nearest neighbor)
6.0-r0-1	works	works but with #4324 and significant dirty refresh issues
6.0.2-r0-1	freezes	freezes and has #4324

the screen freezes exactly before the "xpra shadow server ready" notification is shown in the server. (note that client mouse and keyboard events continue to be forwarded to the sever normally.)

[totaam]

6.0.x are not supported versions: https://github.com/Xpra-org/xpra/wiki/Versions

significant dirty refresh issues

Partial workaround: #4319 (comment)

[Lanchon]

well i just wanted to do a gross bisection regarding this freezing issue.

it was introduced between 6.0 and 6.0.2.

[totaam]

Thanks, I'm not seeing any freezing here - but I'll try again.

[Lanchon]

try linux mint 22 cinnamon, default session. im using that on both sides.

[totaam]

5.0.9-r0-1 works (bilinear interpolation) works (nearest neighbor)

Oh, and BTW, I don't see how that's possible unless your client display is also using some quite odd scaling settings.
The opengl renderer uses linear: #4324 (comment)

[Lanchon]

yes, technically it's actually called bilinear, given that it acts on a 2D surface and has to interpolate linearly in 2 axes, but we are talking about the same thing.

[totaam]

technically it's actually called bilinear

What is surprising here is that you're saying that the non-opengl client uses linear and the opengl client does not.
Both should be using server-side downscaling, which means that the client-side rendering should be irrelevant - and identical.

[totaam]

Found a major issue with opengl scaling, which I still don't fully understand but is finally fixed: 775d435

I believe that this is the same issue that you reported here as it requires desktop-scaling to trigger.
The updates are being sent to the client, but the opengl client was somehow failing to update the window front buffer.

Feel free to re-open if 6.1.2 doesn't fix things for you.
(there will be 6.2 builds in the beta area soon too)

[Lanchon]

What is surprising here is that you're saying that the non-opengl client uses linear and the opengl client does not.
Both should be using server-side downscaling, which means that the client-side rendering should be irrelevant - and identical.

that is correct.

i hope you don't need to see this with your own eyes, i have tested it repeatedly. i also saw that opengl mode was more power efficient on the client so i wanted to use that, but i couldn't because scaled down text is poorly readable without proper filtering.

keep in mind that this is a 5.x client talking to a 6.x server, so it's probably a setup you don't care much to debug.

Both should be using server-side downscaling

how does the 6.x server do the downscaling? this is an operation that would benefit from GPU usage. even if today's CPUs can do it, it'd be much more efficient to do it in the GPU. this also applies to image/video encoding of course. does the server use the GPU? can i tell the server NOT to use the GPU? from the client? are you sure --opengl=off does not influence the server?

[Lanchon]

Feel free to re-open

i can't reopen if a team member closed the issue; only you can. (i could reopen it if i had closed it myself though.)

Found a major issue with opengl scaling, which I still don't fully understand but is finally fixed: 775d435

I believe that this is the same issue that you reported here as it requires desktop-scaling to trigger. The updates are being sent to the client, but the opengl client was somehow failing to update the window front buffer.

according to your description, it is not the same bug. unfortunately, as stated in the OP, this issue happens during downscaling even if opengl is disabled in the client. so you can go ahead and reopen this, sorry...

[totaam]

how does the 6.x server do the downscaling?

Typically via libyuv:

xpra/xpra/codecs/libyuv/converter.pyx

Line 623 in 4a5f14e

def scale_yuv_image(self, image: ImageWrapper) -> ImageWrapper:

this is an operation that would benefit from GPU usage

Very much so.

even if today's CPUs can do it, it'd be much more efficient to do it in the GPU.

Not always. The pixel data usually resides in CPU memory.
Also, setting up a GPU context can be expensive, it isn't worth it for small sizes.

this also applies to image/video encoding of course. does the server use the GPU?

Yes, if available.
In that case, it will upload (if needed) the pixel data and do colourspace conversion, downscaling and encoding on the GPU.

can i tell the server NOT to use the GPU?

Sure: --video-encoders=x264,vpx

from the client?

No.
The client has its own switches for GPU decoding.

are you sure --opengl=off does not influence the server?

Yes.

---

Anyway, that turned out to be interesting, as I found lots of small issues along the way:

• building from source is now easier on Linux Mint: 1ffbae2 + a86d582
• the shadow pointer overlay feature (showing the remote pointer location using an overlay) was broken with desktop-scaling, the fix is in 0e445bb
• GStreamer was causing the whole desktop session to crash hard and drop me back to the login screen, so now we have a --no-gstreamer command line option: make it possible to completely disable all of GStreamer at runtime #4330

And the bilinear vs nearest sampling issue is now tracked here: #4326

The bad news is that apart from these issues, I left my connection to the shadow server open for hours and did not experience any other glitches.
(and that's to and from Linux Mint 22 - except for the OpenGL accelerated client which cannot run in VirtualBox, so I used Fedora 40 to connect)

[Lanchon]

The bad news is that apart from these issues, I left my connection to the shadow server open for hours and did not experience any other glitches.

you didn't need to. this bug is extremely reliable, happening 100% of the time and before the "xpra shadow server ready" notification (so, about 2 seconds after starting showing the remote display).

EDIT: disregard all that comes next, i got mixed up :(

can i tell the server NOT to use the GPU?

Sure: --video-encoders=x264,vpx

from the client?

No.

so the difference between your case and mine seems to be that my server is using the integrated intel GPU and yours is not. which points to they issue probably being worked-around by turning off GPU use in the server.

1. is this assessment compatible with the fact that changing the client from 6.x to 5.x also sidesteps the issue? (this requires that using a 5.x client alters the codepaths related to GPU use in the 6.x server.)

2. what can i add to client command xpra shadow ssh:XXX:0 --desktop-scaling=70% --opengl=off to disable GPU usage in the server so i can test the hypothesis?

[totaam]

so the difference between your case and mine seems to be that my server is using the integrated intel GPU and yours is not.

I have no idea where you got this from.
Did you try to force the video encoders as per the previous comment? Did it help?

which points to they issue probably being worked-around by turning off GPU use in the server.

How so?

is this assessment compatible with the fact that changing the client from 6.x to 5.x also sidesteps the issue?

No, it is not.

what can i add to client command

Start your server independently via an ssh session and use the --video-encoders=x264,vpx switch.
Then connect your client as a separate command.

[Lanchon]

so the difference between your case and mine seems to be that my server is using the integrated intel GPU and yours is not.

I have no idea where you got this from.

sorry my bad, in my head i mixed the the bug report regarding start-desktop and this one regarding shadow desktop. so please disregard the previous message.

---

anyway...

on both client and server running v6.1.1-r0 on mint 22, as per the OP, trying this command fails because the screen stops refreshing, as described in the OP:

xpra shadow --opengl=off --desktop-scaling=70% ssh:XXX:0

however if i do this, the screen keeps on refreshing! the bug never shows up... weird:

ssh XXX xpra shadow :0
xpra attach --opengl=off --desktop-scaling=70% ssh:XXX:0
...
xpra stop ssh:XXX:0

so... there seems to be a race condition. take this command:

ssh XXX xpra shadow :0; sleep N; xpra attach --opengl=off --desktop-scaling=70% ssh:XXX:0; xpra stop ssh:XXX:0

in my case, if N >= 5 then the connection works fine. but if N is smaller it tends to stop refreshing the display. the stopping happens exactly as the "xpra shadow server ready" notification is shown in the server, as said earlier. so it's just a question of not connecting before the notification happens.

furthermore, if you don't wait and the connection freezes, simply disconnecting and attaching again fixes the issue. (of course this doesn't work with the xpra shadow ssh:XXX:0-style commands, as they tear the server down immediately on disconnection.)

Did you try to force the video encoders as per the previous comment? Did it help?

so if you add --video-encoders=x264,vpx and remove the sleep, it works fine:

ssh XXX xpra shadow --video-encoders=x264,vpx :0; xpra attach --opengl=off --desktop-scaling=70% ssh:XXX:0; xpra stop ssh:XXX:0

no delay is needed which means the race condition is gone.

---

maybe consider adding something like --server-option="--video-encoders=x264,vpx" to allow passing arguments to the server.

[totaam]

so if you add --video-encoders=x264,vpx and remove the sleep, it works fine:

So the problem is very likely to be related to gstreamer or some other hardware accelerated encoder.
v6.2.x has a --no-gstreamer command line option for this purpose: #4330

maybe consider adding something like --server-option="--video-encoders=x264,vpx" to allow passing arguments to the server.

Most command line options are already forwarded to the server automatically, without the need for a server-option, unless they are listed in:

xpra/xpra/scripts/config.py

Line 786 in 80a3ba3

CLIENT_OPTIONS: list[str] = [

The problem with encoding options (like video-encoders, etc) is that the command line parsing will replace the defaults (ie: all) with the actual values to be used (ie: h264, vp8, ...) but this substitution is only valid on the client, as the server may have completely different modules installed.
Changing this is very much non-trivial.

[Lanchon]

hi,

tested on v6.2.0-r0:

• xpra shadow ssh:XXX:0 --desktop-scaling=70%
  now works, well done! but there is a different issue: below the local cursor (which correctly echoes the remote cursor shape), xpra also shows a time-lagged remote cursor, but this extra cursor is corrupted.

  • this happens even without desktop scaling.
  • adding --opengl=off makes the issue go away, but results in constantly high cpu usage, so i prefer the defective cursor.

• ssh XXX xpra shadow --video-encoders=x264,vpx :0; xpra attach --desktop-scaling=0.7 ssh:XXX:0; xpra stop ssh:XXX:0
  now became completely unusable due to extreme artifacts. the corrupted remote cursor issue also appears here.

  • adding --opengl=off does not fix the artifacts, but fixes the cursor.
  • disabling desktop scaling fixes the artifacts.
    • but adding --opengl=off on top, though this fixes the cursor, it introduces large black rectangular defects.
  • EDIT: i also tested --encoding=stream, hoping a simple video stream would get rid of the defects, but it made no change.

in all, the original bug report is fixed when --video-encoders=x264,vpx is not used, so you can close this issue if you want.

thanks!