Introduce Windows WSL implementation of podman machine

[n1hility]

Introduce Windows Support for Podman Machine using WSL (Draft)

This PR introduces a new backend machine type "wsl" for windows support. This backend maps each machine instance to a WSL distribution. While this is similar to a virtualized guest utilized by the qemu backend there are a number of architectural differences. Namely, a WSL distribution is essentially a privileged container running under a shared singular WSL Linux kernel covering all other distributions associated with a Windows user. Additionally WSL shares resources with the host Windows OS, so a typical partitioned virtual system config is not utilized. While the architecture adds restrictions, the approach offers increased performance and improved interoperability over a classic windows virtualization setup. As an example of the latter, network and filesystem integration are provided by default to all WSL distributions.

Requirements

Since this implementation is WSL based it requires Windows 10 Build 19041 or later

Fedora Instead of CoreOS Based

Unfortunately, the WSL and CoreOS architectures are not a clean fit at the time of this writing. CoreOS expects to be in full control of the system, executing in a pivoting ramfs boot, governing disks, networking devices, mounts, and managing both the kernel and the underlying OS. Likewise, WSL expects to be in full control of a shared system, including mounts, network interfaces, the WSL Kernel, and even in the init process of the OS (similar to a container). Fedora on the other hand maps cleanly, as it is already the basis of a container distribution. In the future, a specialized WSL bootstrap could be contributed to the CoreOS project, opening the door to unify the OS choice between the qemu and wsl backends.

Nested namespace for Systemd

Since WSL has a special init process that takes PID 1, on machine start this implementation creates a nested process namespace via unshare, and bootstraps systemd under it. As is the case with the qemu backend, both rootful and rootless podman (via linger) are configured. machine ssh enters directly into the system. A user may also enter via the wsl command and the integrated windows terminal support. Upon wsl login, /usr/local/bin/enterns is ran to enter the user into nested namespace. A user may also use the direct command support of the wsl command by prefixing their command with enterns. (e.g. wsl enterns systemctl status). However, the recommended approach is to enter via machine ssh since this requires less steps with sshd already running in the namespace

Low-touch Initiatilization & Automatic WSL Enablement

After running the podman remote windows installer (already in place today), installing a machine instance is similar to the machine workflow on Mac and Linux, simply run (podman machine init). If WSL is not yet installed on the Windows instance, this implementation will perform a WSL installation and relaunch the init process after reboot. The init process downloads Fedora container tarball, imports a WSL distribution, perform a dnf package update, installs podman and dependencies, generates keys, and configures services with a result similar to the ignition output on the qemu based backend. Since the Fedora container image does not include container tools packages, the setup process takes a few minutes. In the future, if a Fedora container image is produced that includes them, this would change to an installation time similar to the CoreOS based qemu backend.

Networking

Networking works out of the box with this implementation since WSL forwards traffic by default. This includes the ability to bind privileged ports on the host side (since Windows allows this). Due to this setup, this PR configures podman to use slirp4netns. Proxying Windows ingress traffic to the machine instance is also possible but requires a special setup on the Windows host to enable a port proxy.

Known Issues

Remote environment variables not yet supported

The current env parsing code in podman remote does not yet support windows. For run -e to work, this needs to be implemented. My plan is to address this in a separate PR

Windows FS mounts not yet supported

While the WSL instance automatically mounts the Windows drive in all distributions, there is a path translation issue between the windows podman remote and the Linux podman service that needs fixing for this to work. I have not yet had time to look into a fix.

Docker API Proxying not yet implemented

Supporting docker API proxying via the standard windows pipe will be added in a subsequent PR

Remaining PR TODO

• Merge of Add image and repo defaults for podman machine common#852
• Testing Strategy?
• Windows 11 Testing
• Verify Mac/Linux since they were touched by this PR

I need guidance on what level of testing you would like to see to promote this PR out of draft.

[vrothberg]

Impressive work, thank you.

[TomSweeneyRedHat]

Wowsa! Looks good overall, thanks @n1hility !

[n1hility]

Wowsa! Looks good overall, thanks @n1hility !

Thanks Tom!!

[n1hility]

Thanks everyone for the detailed reviews. I have updated the PR to your first pass of notes. Right now it is using forks and one-off branches for storage and common until there is a release for both and I can clean that up.

[n1hility]

@n1hility Could we vendor changes into c/common instead of using fork

Sorry about that. I think I somehow backed out the switch in a rebase. This is now fixed

[n1hility]

Thanks for all of the feedback @flouthoc ! I posted replies on the nits, let me know if you disagree with any of the explanations

[rhatdan]

Great work @n1hility
/lgtm
Now we got to get people playing with this and trying it out.

@TomSweeneyRedHat @fatherlinux Could you try this out, now that it is in the main branch.

[rhatdan]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: n1hility, rhatdan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [rhatdan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[flouthoc]

@n1hility Extremely sorry for late reply I was not able to open my workstation for last two days.

Thanks for all of the feedback @flouthoc ! I posted replies on the nits, let me know if you disagree with any of the explanations

Yes thank you so much. LGTM

[thangchung]

Hi all. Has it been released in any version of Podman? We are waiting for this fixing to resolve other issues I have posted in other repo related to #12056 and dotnet/tye#1255

[rhatdan]

No we are looking to release in 4.0 in February.

[rhatdan]

RC1 should be cut in the next few days.

[TomSweeneyRedHat]

@thangchung as @rhatdan noted, it should be available once 4.0 RC1 releases in the very near future. Hopefully by next week.

If you'd like to play with it before hand, you could clone the repository, build Podman, and the msi file should be built that would work for you.

[thangchung]

@rhatdan, @TomSweeneyRedHat thanks for your reply on this. But could you tell me where can I find the build steps for Windows? Or do I need to build it on Ubuntu? I look up several places in this repo, but couldn't find it :|

[thangchung]

@rhatdan @TomSweeneyRedHat I just handled to build with makefile on Ubuntu for the remote client. But whenever I configure and run it on Windows 11 (my machine). It threw the same error as I have got as below

This is podman (after building it, I copy podman.exe to my host - Windows 11)

Then, I run the sock binding on my host machine

> podman system connection add wsl --identity C:\Users\<my_username>\.ssh\id_rsa_localhost ssh://<my_username>@localhost/mnt/wslg/runtime-dir/podman/podman.sock

I can use podman ps correctly on my host machine

But when I run the container from postgres image as the following script

> podman run -e POSTGRES_USER=lab -e POSTGRES_PASSWORD=P@ssw0rd -e POSTGRES_DB=mydb -p 5432:5432 --name postgres_8f9d434a-d --restart=unless-stopped --network tye_network_84e68c7d-5 --network-alias postgres postgres:12-alpine

It threw the exception

That means podman client was still not be able to pass -e into the podman machine on the Ubuntu box (wsl2 on my machine)

/cc @tungphuong

[openshift-ci]

@thangchung: GitHub didn't allow me to request PR reviews from the following users: tungphuong.

Note that only containers members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

@rhatdan @TomSweeneyRedHat I just handled to build with makefile on Ubuntu for the remote client. But whenever I configure and run it on Windows 11 (my machine). It threw the same error as I have got as below

This is podman (after building it, I copy podman.exe to my host - Windows 11)

Then, I run the sock binding on my host machine

> podman system connection add wsl --identity C:\Users\<my_username>\.ssh\id_rsa_localhost ssh://<my_username>@localhost/mnt/wslg/runtime-dir/podman/podman.sock

I can use podman ps correctly on my host machine

But when I run the container from postgres image as the following script

> podman run -e POSTGRES_USER=lab -e POSTGRES_PASSWORD=P@ssw0rd -e POSTGRES_DB=mydb -p 5432:5432 --name postgres_8f9d434a-d --restart=unless-stopped --network tye_network_84e68c7d-5 --network-alias postgres postgres:12-alpine

It threw the exception

That means podman client was still not be able to pass -e into the podman machine on the Ubuntu box (wsl2 on my machine)

/cc @tungphuong

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[n1hility]

@thangchung yes #12056 requires a separate enhancement. This is on my todo for this week.

[n1hility]

@thangchung BTW normally you shouldn't need to add the connection, it should be added automatically by machine init, and picked up from a previous machine init unless you cleared out your connections

[n1hility]

@thangchung Ill ping you as soon as the PR lands if you want to try/test it

[thangchung]

Yes. When everything is okay, please help to let me know. I'm willing to support testing it on Windows 11/WSL2 on the remote client

[n1hility]

Yes. When everything is okay, please help to let me know. I'm willing to support testing it on Windows 11/WSL2 on the remote client

@thangchung I have a PR for this now if you want to try it out: #12928

[thangchung]

@n1hility Just check out the main branch on this repo, build the remote client for Windows and I would like to confirm that it worked perfectly. Thank you guys to make it work for Windows

[tungphuong]

Confirmed, it has worked well. I tried with Tye, perfect now. Thanks!

[rorshik]

Windows 11. Environment variables are not passed. The latest stable versions of wsl and podman 4.5.0 are used