wayland server backend

[totaam]

We should be able to plug into wayland and provide remote access for it.

Reading this freerdp implementation, it doesn't look too hard.
The main difficulty may be in glueing the C api with our (mostly) python server code.

What makes this now more interesting than before is the availability of sub-surfaces - which would allow us to get YUV data for window's sub-regions (like an embedded video player in a browser), without effort.

[totaam]

More links:

• libweston-desktop: *The goals of libweston-desktop are to make it easier to bring-up compositors and also to make it easier to support future versions of XDG_Shell. *
• swc: A library for making a simple Wayland compositor
• wlc: Wayland compositor library
• tutorials: wayland-compositor: minimal wayland compositor
• [https://fedoraproject.org/wiki/Wayland_features#remote_display]: remote display for wayland

[totaam]

Recent discussion on wayland-devel with links:

• Remote display with 3D acceleration using Wayland/Weston: Therefore no support for hardware accelerated OpenGL gets advertised to clients, and clients fall back to software GL.. The hardest part in adding the support to the RDP-backend is implementing the buffer content access efficiently
• Remote display with 3D acceleration using Wayland/Weston: but Waltham does no good if you are already going to use VNC protocol or RDP or any other existing protocol.
• waltham: Waltham IPC Library : Waltham is a network IPC library designed to resemble Wayland both protocol and protocol-API wise. - probably not useful for us.

[totaam]

New wiki: Wayland Remoting. It looks nowhere near ready yet.

[totaam]

This would also solve #510

See:

• Writing a Wayland Compositor, Part 1: Hello wlroots
• Fedora : WaylandRemoteDesktop (still VNC-like!)
• Screen Sharing Support In WebRTC For Wayland Sessions
• Writing a basic compositor using libweston-desktop - user questions don't seem to be getting a lot of answers on the mailing list, and with the complete lack of documentation... (see The situation with Wayland need to change)
• Add support for screen sharing with PipeWire on Wayland
• Building Wayland desktop components with layer shell: A desktop that doesn't work is a desktop that is not going to get used, security doesn't matter if nobody is using it

[totaam]

Note: it is already possible to use a wayland server as an X11 vfb for xpra. ie: based on #1656#comment:12 with weston:

weston --no-config --socket=wayland-30 &
export WAYLAND_DISPLAY=wayland-30
Xwayland :30 -noreset &
xpra start-desktop :30 --use-display --start=lxsession
xpra attach :30

[totaam]

As per Wayland misconceptions debunked: Things like sending pixel buffers to the compositor are already abstracted on Wayland and a network-backed implementation could be easily made.. I'm not sure it is really that easy: the handling of pixel buffers is just one small part of what we do.
The tone of this post is feisty! The problem is that no one seems to really care: all of the people who want network transparency drank the anti-Wayland kool-aid instead of showing up to put the work in.
Yes, more work for us. Yay. Rejoice!
Anyway, since they're willing to help, it's worth looking into: If you want to implement this, though, we’re here and ready to support you! Drop by the wlroots IRC channel and we’re prepared to help you implement this

[totaam]

For native client support, see #2243.
For keyboard mapping support, see #1049 and #2368

[totaam]

The Wayland Protocol.

[totaam]

We're in the same boat as barrier, though there are now virtual device methods? wlroots based VNC server and the protocols it uses virtual_keyboard_unstable_v1 and wlr_virtual_pointer_unstable_v1
(but "unstable" and v1.. been burnt by wayland v1 before)

[totaam]

This is for a secondary virtual screen, but maybe the code to use is the same?
wayvnc : Create virtual screen to use as second screen

synergy: wlr-synergy-client.c

[rohitdesaidev]

Hi it's been quite short time I have been using Xpra and html client. I would like to explore more on wayland side and wanted to launch html5 client on wayland. Is it supported by current xpra ? Any information will be appropriated.

[totaam]

I would like to explore more on wayland side and wanted to launch html5 client on wayland. Is it supported by current xpra ?

Assuming that you mean running wayland applications via xpra. Not yet, no. Sorry.

[totaam]

OBS Studio on Wayland: gs_texture_create_from_dmabuf could be useful to us: we can grab GPU buffers without copying them. (can we convert these buffers to CUDA buffers? will this work with NVENC?)

And wlroots seems to have a virtual pointer:
https://github.com/swaywm/wlroots/blob/master/examples/virtual-pointer.c

[Russell-Jones-OxPhys]

r-c-f/waynergy#16 mentions libei https://gitlab.freedesktop.org/libinput/libei being examined for use with the barrier project: blog post here https://who-t.blogspot.com/2021/08/libei-status-update.html . A post on the goals of libei is here https://lists.freedesktop.org/archives/wayland-devel/2020-July/041556.html

[totaam]

More wayland limitations recorded here: #3322 (comment)
(it's not looking good for wayland as a usable environment - far too many serious limitations)

[totaam]

One more: Isolating Xwayland in a VM: In Wayland, things are not so simple. I have so far found no less than four separate Wayland protocols for copying text

[MayeulC]

I'm rather surprised that Waypipe wasn't mentioned already? I've been using in place of X11 forwarding for a few years now, with a lot of success (it tends to work better with apps such as web browsers IMO), the only thing it lacks is an xpra-like session management where you can disconnect and reconnect.

[totaam]

@MayeulC we have waypipe pipewire support in progress here: #3750 (comment)
Many improvements are still needed, including simulating pointer and keyboard events.

But that's not what I had in mind for a wayland server, I want a seamless server, not a desktop session.

[MayeulC]

@totaam I only see reference to pipewire in that comment, not waypipe, so I am a bit confused. Did I not look around your link enough? Is it the right link?

In case it's a confusion, the two names are unfortunately relatively close, but pipewire handles streams (audio, video) and it's what the xdg screencast portal provides to "screen share", while waypipe is a proxy for Wayland clients, serializing data exchange between a Wayland server and clients to send it over the network, then deserializing it. In the repository, there is a link to that helpfully descriptive blog post.

Waypipe does not implement a full Wayland server, but it's almost there, as it keeps track of allocated buffers to efficiently send the difference.
I think one possible way to close the gap would be to have an extra Wayland server (perhaps wlroots-based), locally connect that to waypipe when there are no clients; and when a client connects, transfer the Wayland server state over to the client, and continue talking over xpra/waypipe. Or something similar, as I haven't dug enough. Hopefully, Wayland servers need to maintain very little state that can't be re-generated (buffer sizes, etc).

[totaam]

@MayeulC my bad, I meant pipewire, not waypipe.
As for waypipe, I am well aware of it but I don't see it as a way forward for multi-platform support.

[MayeulC]

Hmm, I see, what is the issue for cross-platform support with waypipe? Endianess? Code portability? The fact that it relies on a Wayland compositor on the client-side? If the latter, I don't think it would be that complex to run a Wayland compositor on Windows and other platforms (famous last words). Though I can see how that differs from the initial goal, making Waypipe a less appealing solution.

I guess I more-or-less answered my own answer, but both writing a Wayland "server-side" for xpra, and a compositor back-end seem to be quite similar goals, and Waypipe's serializing could be re-purposed.

Edit: my pipe-dream would be to add a "compositor handover" Wayland protocol, to be able to detach windows and attach them to another compositor: suddenly take over all your windows, and send them remotely, or use a "synergy-style" interaction to "move" windows (in fact, just the display, as buffers are then sent to another compositor: wayvnc, xpra, or Waypipe; to be sent off) from one computer to another.

[totaam]

Looks like if we want to have a Wayland seamless server (and at this point, I'm still really not sure that's a good idea since so many things are missing from Wayland, because hand-waving reasons) then we may want something like VKMS
And maybe some inspiration from xwayland-run

[ShadowJonathan]

I'm curious, what kind of things do you still see missing from wayland?

[totaam]

I'm curious, what kind of things do you still see missing from wayland?

@ShadowJonathan I can't list them all here, it would take too long and that alone is a worrying sign.
OTOH: no standard for window decorations, and some very dubious "security" (..) decisions:
https://gist.github.com/probonopd/9feb7c20257af5dd915e3a9f2d1f2277#wayland-breaks-window-icons
This whole list is useful - bear in mind that it was written years after Wayland 1.0 was released and everyone was told that Wayland was ready for prime time, so middle click and other glaring missing features had already been fixed by then. (some of us do remember)

The main one is the lack of window placement - and even this PR is not going to be in the core protocol, assuming it ever gets merged in a usable form which is far from guaranteed at this point. Also: #3721

For xpra, handling X11 windows via Wayland means losing a number of protocols and features, for very little gain.
X11 only apps also outnumber Wayland only apps, probably at least 1000 to 1 right now.
Expecting all these working X11 applications to be modified for a desktop environment which accounts for less than 1% of installed workstations (globally) shows that being application / developer friendly is absolutely not a concern for Wayland.
For once, I believe the "embrace and extend" approach would have been much more productive.

If you scour the links above, you will find that anyone that dares question the direction that Wayland is taking is immediately branded a Wayland-hater, or they will be told that they don't understand what they're doing and then be lectured on how they should be doing it (haha!), or that they need to spend X man-years fixing software that works perfectly well now.
That, right there, is a big red flag.

[totaam]

Some links that may be useful:

• transient seat protocol merged a few months ago
• wprs Like Xpra, but for Wayland, and written in Rust.
• Create ext-image-capture-source-v1 and ext-image-copy-capture-v1