Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full containerized OS using containers #4383

Open
Anghille opened this issue Oct 11, 2024 · 16 comments
Open

Full containerized OS using containers #4383

Anghille opened this issue Oct 11, 2024 · 16 comments
Labels
bug Something isn't working

Comments

@Anghille
Copy link

Describe the bug
I am currently trying to setup a full-containerized desktop (remote VM if you will) using xpra in the server-side. Audio is not working, and video is (for now at least) a mess (no full screen, stuck at 1280*720 50Hz, cursor not clicking were it is showing)

To Reproduce

  1. Setup a Docker image from fedora:40
  2. Use the official xpra repo wget -O /etc/yum.repos.d/xpra.repo https://raw.githubusercontent.com/Xpra-org/xpra/master/packaging/repos/Fedora/xpra.repo
  3. Install xpra and xpra-html5, firefox, @xfce
  4. Since Fedora 40 uses pipewire, remove pipewire and install pulseaudio, pulseaudio-utils, pulseaudio-module-x11

Now this is where things get weird :

if I run xpra start-desktop --start-child=xfce4-session --exit-with-children=yes --daemon=no --bind-tcp=0.0.0.0:14500 and checks the logs, pulseaudio complains it is not finding the system-dbus and is failing to run:

failed to initialize Gtk, no display?
2024-10-11 14:11:51,080 created tcp socket '0.0.0.0:14500'
2024-10-11 14:11:51,081 created rfb socket '0.0.0.0:13500'
Socket path '/tmp/.X11-unix/X14500' not found
2024-10-11 14:11:51,117 no uinput module (not usually needed)

X.Org X Server 1.20.14
X Protocol Version 11, Revision 0
Build Operating System:  6.7.3-200.fc39.aarch64
Current Operating System: Linux 0b54ac4012d8 6.6.12-linuxkit #1 SMP Fri Jan 19 08:53:17 UTC 2024 aarch64
Kernel command line: init=/init loglevel=1 root=/dev/vdb rootfstype=erofs ro vsyscall=emulate panic=0 linuxkit.unified_cgroup_hierarchy=1 console=hvc0   virtio_net.disable_csum=1 eth0.IPNet=192.168.65.3/24 eth0.router=192.168.65.1 eth0.mtu=65535 eth1.dhcp vpnkit.connect=connect://2/1999 com.docker.VMID=5bb344b1-9d9b-4d7b-9abf-607b2164b84e
Build Date: 10 April 2024  12:00:00AM
Build ID: xorg-x11-server 1.20.14-35.fc40
Current version of pixman: 0.43.4
	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: "/run/xpra/14500/Xorg.log", Time: Fri Oct 11 14:11:51 2024
(++) Using config file: "/etc/xpra/xorg.conf"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
2024-10-11 14:11:52,137 debug enabled for ['xpra.scripts.server', 'dbus']
2024-10-11 14:11:52,138 debug enabled for ['xpra.scripts.server', 'dbus']
2024-10-11 14:11:52,138 reload_dbus_attributes(:14500) dbus_env_data=
2024-10-11 14:11:52,138 reload_dbus_attributes(:14500) dbus_env={}
2024-10-11 14:11:52,476 debug enabled for ['xpra.server.dbus.start', 'dbus']
2024-10-11 14:11:52,476 dbus_launch='dbus-launch --sh-syntax --close-stderr', current DBUS_SESSION_BUS_ADDRESS=None
2024-10-11 14:11:52,476 start_dbus(dbus-launch --sh-syntax --close-stderr) env={'HOME': '/root', 'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'LOGNAME': 'root', 'LANG': 'C.UTF-8', 'USER': 'root', 'XDG_SESSION_TYPE': 'x11', 'DISPLAY': ':14500', 'CKCON_X11_DISPLAY': ':14500', 'XAUTHORITY': '/root/.Xauthority'}
2024-10-11 14:11:52,481 out(['dbus-launch', '--sh-syntax', '--close-stderr'])="DBUS_SESSION_BUS_ADDRESS='unix:path=/tmp/dbus-AyyCrQVSaC,guid=bafbb2a0de47fb93a061c2d467093228';\nexport DBUS_SESSION_BUS_ADDRESS;\nDBUS_SESSION_BUS_PID=74;\n"
2024-10-11 14:11:52,481 dbus_pid=74, dbus-env={'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/tmp/dbus-AyyCrQVSaC,guid=bafbb2a0de47fb93a061c2d467093228', 'DBUS_SESSION_BUS_PID': '74'}
2024-10-11 14:11:52,481 started new dbus instance: {'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/tmp/dbus-AyyCrQVSaC,guid=bafbb2a0de47fb93a061c2d467093228', 'DBUS_SESSION_BUS_PID': '74'}
2024-10-11 14:11:52,487 debug enabled for ['xpra.platform.posix.gui', 'posix', 'dbus']
2024-10-11 14:11:52,493 debug enabled for ['xpra.gtk.error', 'x11', 'util', 'error']
2024-10-11 14:11:52,521 debug enabled for ['xpra.server.core', 'websocket']
2024-10-11 14:11:52,522 debug enabled for ['xpra.server.core', 'dbus']
2024-10-11 14:11:52,550 debug enabled for ['xpra.x11.server.base', 'dbus']
2024-10-11 14:11:52,601 debug enabled for ['xpra.net.websockets.common', 'websocket']
2024-10-11 14:11:52,620 debug enabled for ['xpra.net.websockets.handler', 'network', 'websocket']
2024-10-11 14:11:52,621 pointer device emulation using XTest
2024-10-11 14:11:52,622 serving html content from '/usr/share/xpra/www'
2024-10-11 14:11:52,623 wrote pid 7 to '/run/xpra/14500/server.pid'
2024-10-11 14:11:52,637 Warning: socket directory '/run/xpra'
2024-10-11 14:11:52,637  expected permissions 775 but found 700
2024-10-11 14:11:52,639 created unix domain sockets:
2024-10-11 14:11:52,639  '/run/user/0/xpra/0b54ac4012d8-14500'
2024-10-11 14:11:52,639  '/run/xpra/0b54ac4012d8-14500'
2024-10-11 14:11:52,639  '/root/.xpra/0b54ac4012d8-14500'
2024-10-11 14:11:52,639  '/run/xpra/14500/socket'
2024-10-11 14:11:52,639 created abstract sockets:
2024-10-11 14:11:52,639  '@xpra/14500'
2024-10-11 14:11:52,639 init_dbus(74, {'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/tmp/dbus-AyyCrQVSaC,guid=bafbb2a0de47fb93a061c2d467093228', 'DBUS_SESSION_BUS_PID': '74'})
2024-10-11 14:11:52,640 xvfb pid 51
2024-10-11 14:11:52,645 setting dummy crtc and output 0 to:
2024-10-11 14:11:52,645  1280x1024 50Hz (339x271 mm, dpi=96)
2024-10-11 14:11:52,647 monitor 0 is 'VFB-0' 1280x1024
2024-10-11 14:11:52,647 init_dbus_server() dbus_control=True
2024-10-11 14:11:52,647 init_dbus_server() env: {'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/tmp/dbus-AyyCrQVSaC,guid=bafbb2a0de47fb93a061c2d467093228', 'DBUS_SESSION_BUS_PID': '74'}
2024-10-11 14:11:52,647 debug enabled for ['xpra.server.dbus.common', 'dbus']
2024-10-11 14:11:52,652 debug enabled for ['xpra.server.dbus.server_base', 'dbus', 'server']
2024-10-11 14:11:52,652 debug enabled for ['xpra.server.dbus.server', 'dbus', 'server']
2024-10-11 14:11:52,653 debug enabled for ['xpra.x11.dbus.x11_dbus_server', 'dbus', 'server']
2024-10-11 14:11:52,654 org.xpra.Server(<desktop_server.XpraDesktopServer object at 0xffff9d62aa00 (xpra+x11+desktop+desktop_server+XpraDesktopServer at 0xaaab0e8294f0)>)
2024-10-11 14:11:52,654 dbus_exception_wrap() <bound method DesktopServerBase.make_dbus_server of <desktop_server.XpraDesktopServer object at 0xffff9d62aa00 (xpra+x11+desktop+desktop_server+XpraDesktopServer at 0xaaab0e8294f0)>>()=None
2024-10-11 14:11:52,655 debug enabled for ['xpra.notifications.common', 'dbus', 'notify']
2024-10-11 14:11:52,656 debug enabled for ['xpra.dbus.notifications_forwarder', 'dbus', 'notify']
2024-10-11 14:11:52,656 notifications: bus name 'org.freedesktop.Notifications', request=1
2024-10-11 14:11:52,656 D-Bus notification forwarding is available
2024-10-11 14:11:52,663 pulseaudio server started with pid 84
2024-10-11 14:11:52,664  private server socket path:
2024-10-11 14:11:52,664  '/run/xpra/14500/pulse/pulse/native'
2024-10-11 14:11:52,667 debug enabled for ['xpra.gstreamer.common', 'audio', 'gstreamer']
2024-10-11 14:11:52,668 debug enabled for ['xpra.audio.gstreamer_util', 'audio', 'gstreamer']
W: [pulseaudio] main.c: This program is not intended to be run as root (unless --system is specified).
E: [pulseaudio] core-util.c: Failed to connect to system bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
E: [null-sink] util.c: Failed to connect to system bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory

I supposed, based on this ticket (#3212) and an other one that is older where it was stated that 'pulseaudio' and all application where using their own "private" dbus to avoid messing with maybe currently running dbus.

Since we are in a container, I supposed that it needed at least the system dbus to be running. I therefore appended this command : dbus-uuidgen > /var/lib/dbus/machine-id & dbus-daemon --system & && xpra start-desktop --start-child=xfce4-session --exit-with-children=yes --daemon=no --bind-tcp=0.0.0.0:14500 to first run a system dbus on the container.

After that, pulseaudio stopped complaining about missing system dbus. It was complaining about missing module-x11 but a simple dnf install pulseaudio-module-x11 fixed the error. Now I am stuck with :

E: [pulseaudio] module-x11-xsmp.c: Failed to open connection to session manager: Authentication Rejected, reason : None of the authentication protocols specified are supported and host-based authentication failed
E: [pulseaudio] module.c: Failed to load module "module-x11-xsmp" (argument: "display=:14500.0 xauthority=/root/.Xauthority session_manager=local/unix:@/tmp/.ICE-unix/134,unix/unix:/tmp/.ICE-unix/134"): initialization failed.

Any command, with XDR_RUNTIME_DIR manually set to a bunc of places (/run/xpra, /run/user/&{UID}/xpra, /tmp, /home/user/.xpra) fails with:

pactl info
Connection failure: Connection refused
pa_context_connect() failed: Connection refused

Funnly, cloning the xpra repo inside the containers and running the pactl_impl.py return sme informations:

./xpra/audio/pulseaudio/pactl_impl.py
2024-10-11 14:31:46,729 Warning: failed to query pulseaudio using 'pactl info'
2024-10-11 14:31:46,729  Connection failure: Connection refused
2024-10-11 14:31:46,729  pa_context_connect() failed: Connection refused
* device
* devices                         : 0
* pulseaudio
  - cookie-hash                   : <hash>
  - found                         : True
  - id                            : 0@2e8374a74ec0a3dcccf9e678670931bf/84
  - server                        : {2e8374a74ec0a3dcccf9e678670931bf}unix:/run/xpra/14500/pulse/pulse/native
  - wrapper                       : pactl

A lot (if not all) xfce plugins complains with errors of the like GetManagedObjects() failed: org.freedesktop.DBus.Error.Spawn.ChildExited: Launch helper exited with unknown return code 1

For the video side, the xpra always starts with :
2024-10-11 14:11:52,645 setting dummy crtc and output 0 to:
2024-10-11 14:11:52,645 1280x1024 50Hz (339x271 mm, dpi=96)
2024-10-11 14:11:52,647 monitor 0 is 'VFB-0' 1280x1024

I tried using dnf install xrandr && xrandr --output DUMMY0 --rate 60 --size 1920*1080 or other variations, but the browser wont acknoledge this. In fact, all I want is the OS to dynamically adapt to the size of the browser window :(

System Information (please complete the following information):

  • Server OS: Fedora 40 (container version) running on Macos (but same problem on Host Linux ubuntu 22.04.4)
  • Client OS: Macos 15
  • xpra-html5-16.2-1.r0.fc40.noarch
  • xpra-filesystem-6.1.3-10.r0.fc40.noarch
  • xpra-common-6.1.3-10.r0.fc40.aarch64
  • xpra-codecs-nvidia-6.1.3-10.r0.fc40.aarch64
  • xpra-codecs-6.1.3-10.r0.fc40.aarch64
  • xpra-client-6.1.3-10.r0.fc40.aarch64
  • xpra-x11-6.1.3-10.r0.fc40.aarch64
  • xpra-audio-6.1.3-10.r0.fc40.aarch64
  • xpra-client-gtk3-6.1.3-10.r0.fc40.aarch64
  • xpra-server-6.1.3-10.r0.fc40.aarch64
  • xpra-6.1.3-10.r0.fc40.aarch64

Additional context

  • I am trying to create a self-containerized OS that can run on a lot of plateform without the need to setup too much things (docker, and that about it)
  • I am trying to make this container as secure as possible (non-root user, acces to only its home and some other folders + applications)

If you could explain how things works that would help me so much. I understand what dbus is for, what pulseaudio is, what xpra is. But I have a hard time undnerstanding how everything talks to each other nicely :(
I did read A LOT of issues in this repo (Almost every issue, from issue that could be sort-of related to ones that dont even link remotely to my problem) but I am still unsure how things work and why I am running into those problems

@Anghille Anghille added the bug Something isn't working label Oct 11, 2024
@totaam
Copy link
Collaborator

totaam commented Oct 11, 2024

I understand what dbus is for, what pulseaudio is, what xpra is. But I have a hard time undnerstanding how everything talks to each other nicely :(

AFAIK, none of these are absolutely essential, though some applications may misbehave without dbus.
If you don't care about audio, just disable it: --no-audio.

Well done for figuring out how to get pulseaudio installed instead of pipewire.
We should improve the documentation on that subject.

For the video side, the xpra always starts with :
2024-10-11 14:11:52,645 setting dummy crtc and output 0 to:
2024-10-11 14:11:52,645 1280x1024 50Hz (339x271 mm, dpi=96)
2024-10-11 14:11:52,647 monitor 0 is 'VFB-0' 1280x1024

The initial resolution when starting a desktop session can be changed, ie: xpra desktop --resize-display=1080p

I tried using dnf install xrandr && xrandr --output DUMMY0 --rate 60 --size 1920*1080

The resolution is meant to be changed in response to changes to the browser window dimensions, not directly via xrandr.
Works-for-me(tm).


BTW, you don't need to install these in a container:

  • xpra-codecs-nvidia
  • xpra-client-gtk3

@Anghille
Copy link
Author

I do need the audio. The problem is that even with a system dbus lqunched and pulseaudio installed instead of pipewire, audio isnt working at all :(

For video, I will look into it, since it is absolutly ignoring my resolution settings and absolutly not doing dynamic scaling. That is why I tried to use xrandr to fix this issue without success

@totaam
Copy link
Collaborator

totaam commented Oct 12, 2024

Any command, with XDR_RUNTIME_DIR manually set to a bunc of places

So you're running as root inside the container?
It may be better to use a non-root user with a correct $XDG_RUNTIME_DIR set by the login / system.
This may well be what is preventing pulseaudio / dbus from working properly.

You may also want to set XPRA_PRIVATE_PULSEAUDIO=0:

PRIVATE_PULSEAUDIO = envbool("XPRA_PRIVATE_PULSEAUDIO", True)

Since the xpra instance won't be interfering with another one since there aren't any.

not doing dynamic scaling.

What sort of scaling are you expecting it to do?

For the video side, the xpra always starts with :

Oh, it just occurred to me that what you mean by "video" is the resolution of the virtual display.
In the context of xpra, video often means video stream encoding of window contents (for better compression).
"Virtual display", "vfb" or graphics are slightly less ambiguous terms.

@psobolewskiPhD
Copy link

On a related note, is there an official xpra container available in any registry?

@totaam
Copy link
Collaborator

totaam commented Oct 14, 2024

is there an official xpra container available in any registry?

Not yet, we should make one as this would be trivial. (based on Fedora)

@Anghille
Copy link
Author

For the container side, I am running as a non-root user (the multiple warnings in the logs hint clearly in that direction, and I want my image to be as secure as possible). Therefore, I have given the permission to run dbus-daemon --syste command in the sudoers file for the container user.

What I do is RUN as root, installed the aforementioned packages (xpra, xpra-html, pulseaudio ...), setup configuration files (i have copied the files found in /etc/xpra/conf.d/{05_features.conf, etc.}, then run the dbus-daemon --system && xpra [...].

I first tried to run "vanilla" ( modifications to the `05_features.conf, 20_audio.conf etc.), then started tinkering in the files to try to find my problem. I am now trying to run pulse audio with this config:

#!/usr/bin/pulseaudio -nF

### Create virtual output device sink
load-module module-null-sink sink_name=audio_output sink_properties=device.description="Virtual\ Audio\ Output"

# Allow pulse audio to be accessed via TCP (from localhost only), to allow other users to access the virtual devices
load-module module-native-protocol-unix socket=/tmp/pulseaudio.socket auth-anonymous=1

### Make sure we always have a sink around, even if it is a null sink.
load-module module-always-sink

And this overwride command: /usr/bin/pulseaudio --log-level=info --disallow-module-loading --disallow-exit --exit-idle-time=-1 but without any succes.

I am now rebuilding the image, adding an XDG_RUNTIME_DIR set to /run/user/${UID} and the XPRA_PRIVATE_PULSEAUDIO=0

I will try to use more precise wording. I was indeed speaking about the virtual display resolution :)

@totaam
Copy link
Collaborator

totaam commented Oct 14, 2024

Therefore, I have given the permission to run dbus-daemon --system command in the sudoers file for the container user.

As per https://github.com/Xpra-org/xpra/wiki/Reporting-Bugs, this is an essential piece of information, not something to be gleaned later.

then run the dbus-daemon --system && xpra [...].

Does it work if everything runs as root, including xpra?
I would start there.

@Anghille
Copy link
Author

Anghille commented Oct 22, 2024

(still_ on it, just a LOT of things going on, I will keep you posted when I can give you a more thorough description 🔸 ) Thank you for everything !

@Anghille
Copy link
Author

Anghille commented Nov 13, 2024

Just to keep you updated. I have more stuff to share, not sure if it is the right format/place thought. Hope it will be fine for you here:

I manage to make the sound work inside the fedora container. It was in fact my laptop that wasnt able to get the sound for some reason (mac m1 i guess), but other friends (other mac, m1/m2 for different sizes, as well as linux/win computers) confirmed sound was working on their computer so it work I guess.

Once I confirmed the sound was working, I set my goals higher. What I did at first was a simple setup that run dbus-daemon --system && xpra start-desktop [other args]. But this would break the container as any dbus PID would remain between container restart and crash the container at runtime. Since the goal was to create a real OS inside a container, I needed a more permanent and reliable solution:

I now use s6-overlays to start a process 1 pid that manages other services (xpra included). s6-overlay is kind of a lightweight systemd (kind of). I wont go into details as to how it works, but basically, I have a dbus-daemon service that run a system dbus, which fixed every error I had about desktop stuff (widget that couldnt not contact other dbus endpoints path, error with pulseaudio dbus ...), an init-session service that is used as a one shot service to fix permissions and other necessary files that exists only at runtime, and the xpra service itself, dependant of the 2 other services.

Just remember : Some elements cannot be set inside the Dockerfile since it will be overwritten at runtime (aka. when the container is started). For exemple, the services defined for s6 are executed AT-RUNTIME. Which means you can defined iptables, /etc/resolve.conf, /etc/hosts, and any other stuff as one-shot services to setup your environment correctly. For exemple, when the container is started, all ENV key=val is setup as en envrionement variable in the container. You can get all those env variable at runtime, and write them inside a file. In the other hand, you configure xpra to read this file and load all env variable inside it and this way, you managed to pass-down env variables without having to put this in a file, and load it. Some might argue that the latter is more "container-friendly" thought.

How does this works:

Overall

  1. s6 uses the files found in /etc/s6-overlay/s6-rc.d/user/contents.d/ to know which services are enabled (a simple file that has the same name as the service is just what you need. Nothing inside. exemple : touch /etc/s6-overlay/s6-rc.d/user/contents.d/service-1 )
  2. It load in a runtime directory /run/s6-rc/services (if I am not mistaken) the services configurations found at `/etc/s6-overlay/s6-rc.d/{service-1/, service2/, ...}
  3. Services can have a dependency tree (service-2 must wait for service-1, that waits for s6-base), and are of 2 types (longrun, aka process, or oneshot, aka script). In my setup, I have an init-session of type oneshot that fixes runtime permissions and directories, then the dbus-system longrun service is started, then xpra service is started.

Inside a service directory

I encourage to read the s6-overlay github but basically, all services container a run file which is executable (imagine this as your bash script that exec your process), or a up file for oneshot service type. You can configure readiness (checks) probes if services are not using file-descriptor to tell if they are ready or not. Any finish script is used when the service crashes -> s6 run this script before trying to put the service back up again.

DBUS

A system dbus seems to be necessary for a desktop environment to work properly. I therefore use the (lazy) allow-all.conf below for my system dbus (since it is run inside the container, it is not a security risk for the host system dbus itself):

/etc/dbus-1/system.d/allow-all.conf

<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
    <!-- Allow all actions for all users and services -->
    <policy context="default">
        <allow user="*" />
        <allow send_destination="*" />
        <allow send_interface="*" />
        <allow own="*" />
        <allow receive_type="method_call" />
        <allow receive_type="method_return" />
        <allow receive_type="signal" />
        <allow receive_type="error" />
    </policy>
</busconfig>

and run the dbus with this command:

exec exec s6-notifyoncheck -s 800 -w 2000 -t 5000 -n 3 dbus-daemon --system --nofork --syslog --address="${DBUS_SYSTEM_BUS_ADDRESS}"

where DBUS_SYSTEM_BUS_ADDRESS=unix:path=/usr/dbus/system_dbus_socket, which is specified as an ENV variable inside the Dockerfile.

Once again, any related s6 overlay is not explain in details here (unless @totaam asks for it, but I dont want to polute the issue too much).

XPRA specific setup

xpra is run inside the container as an s6 longrun service, using exec s6-setuidgid ${TARGET_USER} xpra start-desktop :22 --start=xfce4-session --daemon=no --bind-tcp=0.0.0.0:14500 --gid=1000 --uid=1000 --resize-display=1920x1080

(s6-setuidgid ${TARGET_USER} is an s6 wrapper than enable the user to run a command using the specified user. Why not using 'USER XXXdirectly in the dockerfile orsu - XXXin therunscript of xpra is outside the scope of this explanation and more related tos6` functionalities).

I use xfce as it is more lightweight than a full-fledge plasma or gnome desktop environment.

⚠️ Currently the auto-resize of the display is not working properly : xpra correctly detects the resolution (when putting the browser window full screen then windowed , moved around..) and update the DUMMY0 display resolution, but for some reason xfce won't change the UI resolution to follow this change. I tried plasma kde which displays the same behavior. I therefore needed to fix the display to 1920x1080 using xpra --resize-display which is not ideal (since my main screen is 4k and my laptop has a weird display size

Note : XPRA tries to isolate from the host. Which means any ENV variable defined in dockerfile is not passed to xpra by default. What I do in the init-session is to get all current env variable, write it into /etc/xpra/server.env and run XPRA with the config source=/etc/xpra/server.env

For reference, here is the small pipe commands that takes all Dockerfile env at runtime, and load it inside this file :

printenv | grep -E '^[A-Za-z_][A-Za-z0-9_]*=' | grep -v '^_=' | sed 's/^/export /' > /etc/xpra/server.env

Dockerfile

For reference, here is a dockerfile exemple that I use (simplified since I use a LOT more custom scripts and packages for my own needs).

FROM fedora:40

USER root

# Update and install all required packages in a single RUN command to reduce layers
ADD https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-40.noarch.rpm        /tmp
ADD https://raw.githubusercontent.com/Xpra-org/xpra/master/packaging/repos/Fedora/xpra.repo /etc/yum.repos.d/
ADD https://gitlab.com/paulcarroty/vscodium-deb-rpm-repo/-/raw/master/pub.gpg               /tmp

RUN rpm --import /tmp/pub.gpg && \
    printf "[gitlab.com_paulcarroty_vscodium_repo]\nname=download.vscodium.com\nbaseurl=https://download.vscodium.com/rpms/\nenabled=1\ngpgcheck=1\nrepo_gpgcheck=1\ngpgkey=https://gitlab.com/paulcarroty/vscodium-deb-rpm-repo/-/raw/master/pub.gpg\nmetadata_expire=1h" > /etc/yum.repos.d/vscodium.repo && \
    dnf -y install /tmp/rpmfusion-free-release-40.noarch.rpm && \
    dnf -y update && \
    dnf -y install \
        bind-utils \
        codium \
        d-spy \
        firefox \
        git \
        gnome-terminal \
        iproute \
        iptables-services \
        iputils \
        qt \
        mesa-vulkan-drivers \
        nautilus \
        nodejs \
        openbox \
        python3 \
        python3-pip \
        vim \
        wget \
        which \
        @xfce \
        xpra \
        xpra-html5 \
        xrandr \
        xset \
        zsh && \
    npm install uglify-js -g && \
    # Ensure we are using pulseaudio instead of pipewire
    dnf install -y \
        pulseaudio \
        pulseaudio-utils --allowerasing && \
    dnf clean all && \
    rm -rf /var/cache/dnf

#### #### #### #### #### #### #### #### #### #### #### #### #### #### #### #### #### #### #### #### #### #### ####
#### DESKTOP ENV SETTINGS
####
ARG TARGET_USER=xpra-user
ARG S6_OVERLAY_VERSION=3.2.0.2
COPY --chown=dbus:dbus                                 src/config/dbus/allow.conf                 /etc/dbus-1/system.d/allow.conf
COPY --chown=dbus:dbus                                 src/config/dbus/allow.conf                 /etc/dbus-1/session.d/allow.conf



RUN groupadd -r -g 1000 ${TARGET_USER} && \
    adduser -u 1000 -g 1000 --shell /bin/bash ${TARGET_USER} && \
    chown -R ${TARGET_USER}:${TARGET_USER} /home/${TARGET_USER} && \
    mkdir -p /home/${TARGET_USER}/.config/{tint2,openbox,rofi} && \
    echo "${TARGET_USER}:user-password" | chpasswd && \
    echo "root:root-password" | chpasswd && \
    chown -R ${TARGET_USER}:${TARGET_USER} /home/${TARGET_USER}/.config && \
    # syslog
    groupadd -g 32760 syslog && \
    useradd -u 32760 -g syslog -s /sbin/nologin syslog && \
    groupadd -g 32761 sysllog && \
    useradd -u 32761 -g sysllog -s /sbin/nologin sysllog && \
    # Setup
    dbus-uuidgen > /var/lib/dbus/machine-id && \
    mkdir -p /run/xpra && \
    mkdir -p /tmp/22 && \
    mkdir -p /tmp/.X11-unix && \
    mkdir -p /run/user/1000/xpra/22 && \
    chmod 700 /tmp/.X11-unix && \
    chmod 700 /tmp/22 && \
    chmod 700 /run/xpra && \
    chmod 700 /run/user/1000/ && \
    chmod 700 /run/user/1000/xpra && \
    chmod 700 /run/user/1000/xpra/22 && \
    chmod 644 /home/${TARGET_USER}/.vimrc && \
    chmod 644 /home/${TARGET_USER}/.zshrc && \
    chown -R ${TARGET_USER}:${TARGET_USER} /tmp/22 && \
    chown -R ${TARGET_USER}:${TARGET_USER} /run/xpra && \
    chown -R ${TARGET_USER}:${TARGET_USER} /run/user/1000/ && \
    chown -R ${TARGET_USER}:${TARGET_USER} /run/dbus/ && \
    echo 'load-module module-native-protocol-tcp' >> /etc/pulse/default.pa && usermod -aG audio,pulse,video ${TARGET_USER}

# Set environment variable
ENV DISPLAY=":22"
ENV MESA_LOADER_DRIVER_OVERRIDE=llvmpipe
ENV SHELL=/bin/bash
ENV USER=${TARGET_USER}
ENV HOME=/home/${TARGET_USER}
ENV DBUS_SYSTEM_BUS_ADDRESS="unix:path=/run/dbus/system_bus_socket"
ENV XDG_SESSION_TYPE=x11
ENV XDG_RUNTIME_DIR=/run/user/1000
ENV XDG_CONFIG_HOME=$HOME/.config
ENV XDG_CONFIG_DIRS=/etc/xdg/
ENV XDG_CACHE_HOME=$HOME/.cache
ENV XDG_DATA_DIRS=/usr/local/share/:/usr/share/
ENV XDG_DATA_HOME=$HOME/.local/share

##############################################################################################
# s6-overlay setup
##############################################################################################
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-noarch.tar.xz /tmp
RUN tar -C / -Jxpf /tmp/s6-overlay-noarch.tar.xz
ADD https://github.com/just-containers/s6-overlay/releases/download/v${S6_OVERLAY_VERSION}/s6-overlay-x86_64.tar.xz /tmp
RUN tar -C / -Jxpf /tmp/s6-overlay-x86_64.tar.xz

### S6-overlays (systemctl docker equivalent)
COPY src/s6-overlay/s6-rc.d /etc/s6-overlay/s6-rc.d
COPY src/s6-overlay/scripts /etc/s6/scripts

RUN chmod +x /etc/s6-overlay/s6-rc.d/*/finish \
             /etc/s6-overlay/s6-rc.d/*/run \
             /etc/s6-overlay/s6-rc.d/*/up \
             /etc/s6-overlay/s6-rc.d/*/notification-fd \
             /etc/s6-overlay/s6-rc.d/*/data/check \
             /etc/s6/scripts/* && \
    touch  /etc/s6-overlay/s6-rc.d/user/contents.d/init-session      && \
    touch  /etc/s6-overlay/s6-rc.d/user/contents.d/dbus-system       && \ 
    touch  /etc/s6-overlay/s6-rc.d/user/contents.d/xpra   

EXPOSE 14500
WORKDIR /home/${TARGET_USER}

ENTRYPOINT [ "/init" ]

Where init-session is used to fix runtime directory permissions, and other sutff that are needed for my setup but not relevent for a demo dockerfile. I keep it here for other people reference.

The directory structure for s6

    └── s6-overlay
        ├── s6-rc.d
        │   ├── dbus-system
        │   │   ├── data
        │   │   │   └── check
        │   │   ├── dependencies.d
        │   │   │   └── init-session
        │   │   ├── finish
        │   │   ├── notification-fd
        │   │   ├── run
        │   │   └── type
        │   ├── init-session
        │   │   ├── dependencies.d
        │   │   │   └── base
        │   │   ├── type
        │   │   └── up
        │   └── xpra
        │       ├── dependencies.d
        │       │   └── dbus-system
        │       ├── run
        │       └── type
        └── scripts
            ├── init

with dbus-system/run :

#!/command/with-contenv bash

# Color and icon definitions
RESET="\e[0m"
CYAN="\e[36m"
BLUE="\e[34m"
RED="\e[31m"
YELLOW="\e[33m"
GREEN="\e[32m"

INFO_ICON="${BLUE}[ℹ️ ]${RESET}"
WARNING_ICON="${YELLOW}[⚠️]${RESET}"
ERROR_ICON="${RED}[❌]${RESET}"
SUCCESS_ICON="${GREEN}[✅]${RESET}"

log_separator() {
 echo -e "${CYAN}$(date '+%Y-%m-%d %H:%M:%S') ====[ $1 ]=====================${RESET}"
}

log_success() {
 echo -e "$(date '+%Y-%m-%d %H:%M:%S') ${SUCCESS_ICON} ${GREEN}$1${RESET}"
}

log_info() {
 echo -e "$(date '+%Y-%m-%d %H:%M:%S') ${INFO_ICON} ${BLUE}$1${RESET}"
}

log_warning() {
 echo -e "$(date '+%Y-%m-%d %H:%M:%S') ${WARNING_ICON} ${YELLOW}$1${RESET}"
}

log_error() {
 echo -e "$(date '+%Y-%m-%d %H:%M:%S') ${ERROR_ICON} ${RED}$1${RESET}"
}

# Starting log separator for dbus-daemon Script
log_separator "Starting dbus-daemon service"

# Check for sudo privileges
if ! sudo -v &> /dev/null; then
 log_error "This script requires sudo privileges. Please run as a user with sudo permissions."
else
 log_info "User has sudo privileges"
fi

# Check if dbus-daemon is installed
if ! command -v dbus-daemon &> /dev/null; then
 log_error "dbus-daemon command not found. Please install the dbus-daemon package and try again."
else
 log_info "dbus-daemon is installed"
fi

# Start dbus-daemon in the background and check the status
log_info "Starting dbus-daemon with address: ${DBUS_SYSTEM_BUS_ADDRESS}"
log_info "Setting DBUS_SYSTEM_BUS_ADDRESS environment variable"
exec exec s6-notifyoncheck -s 800 -w 2000 -t 5000 -n 3 dbus-daemon --system --nofork --syslog --address="${DBUS_SYSTEM_BUS_ADDRESS}"

@totaam
Copy link
Collaborator

totaam commented Nov 19, 2024

@Anghille I haven't had time to look into this properly yet as I'm working on making new releases.

OTOH:

  • I would use mkdir -m 700 DIR
  • constify that display number (22), groups (32760), etc
  • why exec exec right at the end?
  • why is sudo needed?

@Anghille
Copy link
Author

Anghille commented Nov 19, 2024

Hi, no worries. You are doing all of this out of goodwill and on your own time. You don't ever need to apologize for the time you take for this project !

  • you're right for the mkdir -m 700 DIR
  • the exec exec was a mistake due to copy-pasting, fixing it in the previous comment
  • 'sudo' is indeed not needed during scripts execution for the services and could be cleaned. It is best imo to keep it in the image itself so users with correct permissions can use sudo (as you would be able to do in a real OS).
  • not sure hat you meant by "constify" in the context of the differents groups, display number etc.

I am not sure what to do with this issue now that the main issue I had with the sound problem is fixed in my end. If it is ok with you, I can either close this (and remove this load out of your mind), keep it open and give a more detail overview of the differents files used so You can use it later if you want to integrate it as an exemple, or I could fork your repository and add the exemple my self and we iterate from here before merging ? Feel free to tell me what works best for you :)

My goal at the end is to have an image that works out of the box with different flavor (xfce, gnome, kde) with and without GPU support (not there yet). If its something that is of interest in this project, I would gladly share exemples as I get them working reliably

@totaam
Copy link
Collaborator

totaam commented Nov 19, 2024

not sure hat you meant by "constify"

I meant defining DISPLAY_NO=22 somewhere, then referring to ${DISPLAY_NO} instead of 22. (and same for the others)

I am not sure what to do with this issue...

I think this is very valuable, and should probably be turned into some kind of documentation and / or base container that people can more easily extend to suit their needs.
That way, everyone can benefit from fixes and improvements.

So, feel free to contribute as little or as much as you like, in whichever way is easier for you.
I can't guarantee that I will have much bandwidth to dedicate to this task in the near future, but this is definitely something that should be improved.

@totaam
Copy link
Collaborator

totaam commented Nov 19, 2024

Whilst we're here. It would be useful to extend or at least document some kind of container GPU acceleration, be it via EGL (similar to https://github.com/selkies-project/docker-nvidia-egl-desktop) or for WSL: https://github.com/Xpra-org/xpra/blob/master/docs/Usage/WSL.md

@Anghille
Copy link
Author

Anghille commented Nov 19, 2024

i have the chance to run a Kubernetes at home with both AMD and NVIDIA GPUs (small ones - a GT1030 and a RX550), I might be able to test / build and deploy a fully containerized image at some point using GPU acceleration. If this is the case, I will gladly share any finding on this repo for reference. I have already run and modified the selkies-project image, especially to understand how it differ from xpra in some points, so I might be able to use this as a starting point and adapt it for XPRA use case

@j-landru
Copy link

j-landru commented Dec 3, 2024

⚠️ Currently the auto-resize of the display is not working properly : xpra correctly detects the resolution (when putting the browser window full screen then windowed , moved around..) and update the DUMMY0 display resolution, but for some reason xfce won't change the UI resolution to follow this change. I tried plasma kde which displays the same behavior. I therefore needed to fix the display to 1920x1080 using xpra --resize-display which is not ideal (since my main screen is 4k and my laptop has a weird display size

Same issue for me in abcdic context (https://github.com/orgs/Xpra-org/discussions/3911), an Alpine X11/SDDM/KDE/Plasma desktop container. Full desktop auto-reszize used to work (context : qt 5.x/Plasma 5.x). Since last update (qt6.x Plasma 6.x)

  • dummy0 virtual screen auto-resizes correctly, applicative windows can adjust at full screen resolution at browser window size
  • but some resources remain at initial resolution 1280x1024 : SDDM background, plasma desktop wallpaper, as well as kde panel ,...

Until now, I haven't found the right way to have again a full desktop auto-resizing.

Context : abcd (Alpine Based Containerized os Derivative) is a kairos full OS container, abcdic (abcd) Inside Container is the the derivative container image run-able inside common container runtime, like sysbox/docker rootless container runtime, as well as podman and maybe Incus/LXD. As an Alpine full OS container, Init is Openrc (not systemd), so service dependencies, like dbus, are driven at init step before SDDM/X11/Xdummy/xpra start.

Additional info : In kde systemsettings panel,

  • "Display Configuration" tab : pointing to DUMMY0 virtual screen, shows and auto adjusts resolution to the browser window size change, (and as told above, common app window can adjust full virtual screen) ;
  • "wallpaper" tab : points to an unmodifiable VFB-0 device at iniitial resolution 1280x1024 (that's probably why plasma background image and kde panel remain sized at that resolution ?)

It's probably not a good practice to publish here some screenshot images...

@j-landru
Copy link

j-landru commented Dec 3, 2024

here server log from xpra inside sysbox/docker rootless container

2024-12-03 13:51:31,958  `paramiko` module not found
2024-12-03 13:51:31,962 created tcp socket '[::1]:8080'
2024-12-03 13:51:31,962 created tcp socket '[::ffff:127.0.0.1]:8080'
2024-12-03 13:51:32,054 no uinput module (not usually needed)
2024-12-03 13:51:32,226 Warning: failed to reload the dbus session attributes
2024-12-03 13:51:32,226  for mode desktop
2024-12-03 13:51:32,226  a new dbus instance will be started
2024-12-03 13:51:32,226  which may conflict with the previous one if it exists
2024-12-03 13:51:32,429 pointer device emulation using XTest
2024-12-03 13:51:32,430 wrote pid 1215 to '/tmp/0-runtime-dir/xpra/0/server.pid'
2024-12-03 13:51:32,431 serving html content from '/usr/share/xpra/www'
2024-12-03 13:51:32,574 Error querying drm devices: -2
2024-12-03 13:51:32,576 some GStreamer elements are missing or unavailable on this system:
2024-12-03 13:51:32,576  vah264lpenc, vah264enc, vaapih264enc, vaapih265enc, x264enc, openh264enc, vp8enc, vp9enc, av1enc
2024-12-03 13:51:32,582 created unix domain sockets:
2024-12-03 13:51:32,582  '/tmp/0-runtime-dir/xpra/abcdic4desktop-0'
2024-12-03 13:51:32,582  '/run/xpra/abcdic4desktop-0'
2024-12-03 13:51:32,582  '/root/.xpra/abcdic4desktop-0'
2024-12-03 13:51:32,582  '/tmp/0-runtime-dir/xpra/0/socket'
2024-12-03 13:51:32,582 created abstract sockets:
2024-12-03 13:51:32,582  '@xpra/0'
2024-12-03 13:51:32,582 xvfb pid not found
2024-12-03 13:51:32,588 setting dummy crtc and output 0 to:
2024-12-03 13:51:32,589  1280x1024 50Hz (339x271 mm, dpi=96)
2024-12-03 13:51:32,589 monitor 0 is 'VFB-0' 1280x1024
2024-12-03 13:51:32,593 Warning: failed to load the mdns publishers
2024-12-03 13:51:32,593  No module named 'zeroconf'
2024-12-03 13:51:32,593  No module named 'avahi'
2024-12-03 13:51:32,593  install 'python-avahi', 'python-zeroconf'
2024-12-03 13:51:32,593  or use the 'mdns=no' option
2024-12-03 13:51:32,617 D-Bus notification forwarding is available
2024-12-03 13:51:32,627 watching for applications menu changes in:
2024-12-03 13:51:32,627  '/usr/share/applications'
2024-12-03 13:51:32,627 using 'Gio file monitor'
2024-12-03 13:51:32,637 Warning: cannot load menu data
2024-12-03 13:51:32,637  No module named 'xdg'
2024-12-03 13:51:32,638 Warning: webcam forwarding is disabled
2024-12-03 13:51:32,638  the virtual video directory '/sys/devices/virtual/video4linux' was not found
2024-12-03 13:51:32,638  make sure that the 'v4l2loopback' kernel module is installed and loaded
2024-12-03 13:51:32,638  or use the 'webcam=no' option
2024-12-03 13:51:32,638 found 0 virtual video devices for webcam forwarding
2024-12-03 13:51:32,672 Warning: vpx encoder testing failed with vp8:
2024-12-03 13:51:32,672  failed to instantiate vp8 encoder with ABI version 34: Invalid parameter
2024-12-03 13:51:32,673 Warning: vpx encoder testing failed with vp9:
2024-12-03 13:51:32,673  failed to instantiate vp9 encoder with ABI version 34: Invalid parameter
2024-12-03 13:51:32,673 vpx: all the codecs have failed! (vp8, vp9)
2024-12-03 13:51:32,676 31.0GB of system memory
2024-12-03 13:51:32,685 xpra is ready.
2024-12-03 13:51:32,685 xpra X11 desktop server version 6.2.1-r0
2024-12-03 13:51:32,686  uid=0 (root), gid=0 (root)
2024-12-03 13:51:32,686  running with pid 1215 on Linux 6.6.58-gentoo-r1-x86_64
2024-12-03 13:51:32,686  cpython 3.12
2024-12-03 13:51:32,686  connected to X11 display :0 with 24 bit colors
2024-12-03 13:51:32,686  initial resolution: 1280x1024
2024-12-03 13:51:32,688   :0.0 (339x271 mm - DPI: 96x96)
2024-12-03 13:51:32,688     VFB-0
2024-12-03 13:51:32,688     VFB0-19334.363990259
2024-12-03 13:51:32,692 Failed to load library ( 'libGLX.so.0' ): Error loading shared library libGLX.so.0: No such file or directory
2024-12-03 13:51:32,693 No OpenGL_accelerate module loaded: No module named 'OpenGL_accelerate'
2024-12-03 13:51:32,718 Failed to load library ( 'libOpenGL.so.0' ): Error loading shared library libOpenGL.so.0: No such file or directory
2024-12-03 13:51:32,718 Error during threaded setup of <class 'xpra.server.mixins.display.DisplayManager'>
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/xpra/server/base.py", line 176, in threaded_init
    c.threaded_setup(self)
  File "/usr/lib/python3.12/site-packages/xpra/server/mixins/display.py", line 71, in threaded_setup
    self.opengl_props = self.query_opengl()
                        ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xpra/server/mixins/display.py", line 81, in query_opengl
    from xpra.opengl import backing
  File "/usr/lib/python3.12/site-packages/xpra/opengl/backing.py", line 18, in <module>
    from OpenGL.GL import (
  File "/usr/lib/python3.12/site-packages/OpenGL/GL/__init__.py", line 4, in <module>
    from OpenGL.GL.VERSION.GL_1_1 import *
  File "/usr/lib/python3.12/site-packages/OpenGL/GL/VERSION/GL_1_1.py", line 14, in <module>
    from OpenGL.raw.GL.VERSION.GL_1_1 import *
  File "/usr/lib/python3.12/site-packages/OpenGL/raw/GL/VERSION/GL_1_1.py", line 7, in <module>
    from OpenGL.raw.GL import _errors
  File "/usr/lib/python3.12/site-packages/OpenGL/raw/GL/_errors.py", line 4, in <module>
    _error_checker = _ErrorChecker( _p, _p.GL.glGetError )
                                        ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'glGetError'

Warning: running as root

2024-12-03 13:51:32,778 GStreamer version 1.24.9
2024-12-03 13:53:46,697 Handshake complete; enabling connection
2024-12-03 13:53:46,714 HTML5 Posix Firefox client version 16.2
2024-12-03 13:53:46,715 setting keyboard layout to 'fr'
2024-12-03 13:53:46,755  client root window size is 1452x937
2024-12-03 13:53:46,755   HTML (384x248 mm - DPI: 96x96)
2024-12-03 13:53:47,259 server virtual display now set to 1452x937
2024-12-03 13:53:47,267  automatic picture encoding enabled, also available:
2024-12-03 13:53:47,268   rgb24, rgb32, png, webp, jpeg
2024-12-03 13:53:47,317 DPI set to 96 x 96
2024-12-03 13:54:13,719 426 packets received (22K bytes)
2024-12-03 13:54:13,719 281 packets sent (22M bytes)
2024-12-03 13:54:13,721 HTML5 client 1 disconnected.
2024-12-03 13:54:14,058 Handshake complete; enabling connection
2024-12-03 13:54:14,059 HTML5 Posix Firefox client version 16.2
2024-12-03 13:54:14,061 keyboard mapping already configured (skipped)
2024-12-03 13:54:14,061  client root window size is 1452x937
2024-12-03 13:54:14,062   HTML (384x248 mm - DPI: 96x96)
2024-12-03 13:54:14,065 server virtual display now set to 1452x937
2024-12-03 13:54:14,073  automatic picture encoding enabled, also available:
2024-12-03 13:54:14,074   rgb24, rgb32, png, webp, jpeg
2024-12-03 13:54:14,122 DPI set to 96 x 96
2024-12-03 16:51:11,787 New tcp connection received
2024-12-03 16:51:11,787  from '::1, 45192, 0, 0'
2024-12-03 16:51:11,787  on '[[::1]]:8080'
2024-12-03 16:51:11,952 17K packets received (0.7M bytes)
2024-12-03 16:51:11,952 13K packets sent (0.6G bytes)
2024-12-03 16:51:11,954 HTML5 client 2 disconnected.
2024-12-03 16:51:12,399 Handshake complete; enabling connection
2024-12-03 16:51:12,400 HTML5 Posix Firefox client version 16.2
2024-12-03 16:51:12,402 keyboard mapping already configured (skipped)
2024-12-03 16:51:12,402  client root window size is 1555x1111
2024-12-03 16:51:12,402   HTML (411x294 mm - DPI: 96x96)
2024-12-03 16:51:12,405  automatic picture encoding enabled, also available:
2024-12-03 16:51:12,405   rgb24, rgb32, png, webp, jpeg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants