Unable to launch containers that require systemd due to use of Alpine (OpenRC) as hypervisor

[Westie]

Description

I'm unable to launch containers on colima that require systemd and mounted cgroup directories, a notable container being freeipa

[root@ipa /]# stat /sys/fs/cgroup /sys/fs/cgroup/systemd
  File: /sys/fs/cgroup
  Size: 300       	Blocks: 0          IO Block: 4096   directory
Device: 0,62	Inode: 1           Links: 15
Access: (0555/dr-xr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2022-07-23 20:57:45.070000200 +0000
Modify: 2022-07-23 20:57:45.070000200 +0000
Change: 2022-07-23 20:57:45.070000200 +0000
 Birth: -
stat: cannot statx '/sys/fs/cgroup/systemd': No such file or directory

Version

Colima Version: colima version 0.4.4
Lima Version: limactl version 0.11.2
Qemu Version: qemu-img version 7.0.0

Operating System

• macOS Intel
• macOS M1
• Linux

Reproduction Steps

1. Launch freeipa, with mountings for /sys/fs/cgroup
2. Watch as it instantly crashes with code 255

Expected behaviour

I would expect freeipa to work.

Additional context

This appears to be related with the switch from ubuntu to Alpine, therefore this is actually a regression. There is a couple of other bugs that this is related to, probably stuff like kubernetes-sigs/kind#2778

[Westie]

Appears to be a hvf issue.

[Westie]

I'd like to reopen this issue as I believe I understand what's actually going on now.

It isn't related to hvf as I had originally thought, but it is related to the migration to Alpine. I'd like to use colima as it provides a much cleaner environment to work with over platforms that do natively(!) support systemd, such as Multipass (which uses various flavours of Ubuntu as its hypervisor)

I do agree with the decision to migrate Colima to Alpine for 99.9% of all use cases as ultimately, OpenRC is better suited to what most people use docker for.

However, I was wondering if it was at all possible for there to be a way to install custom images - I noticed that the internal colima config generation appears to be hard coded to a pinned version of Alpine with no way to suggest a custom image

[abiosoft]

Can you give instructions on how to reproduce the issue? Also, is it something that works on Docker Desktop?

The Ubuntu layer container (colima start --layer) does run systemd and it mounts the cgroup directory. However, it runs privileged and utilises containerd directly instead of docker.

But if cgroups v2 are needed, then it is not supported in the current image.

However, I was wondering if it was at all possible for there to be a way to install custom images - I noticed that the internal colima config generation appears to be hard coded to a pinned version of Alpine with no way to suggest a custom image

The Alpine image is custom with bundled dependencies, simply swapping it with Ubuntu wouldn't work.

[BenTheElder]

kubernetes-sigs/kind#2778 (comment)

TLDR the majority of the container ecosystem really only tests under systemd and I don't recommend OpenRC for running container hosts

[BenTheElder]

I also wouldn't recommend musl + containers due to the dns resolver though wolfi does modified alpine with glibc I think.

[abiosoft]

I am currently exploring the possibility of enabling cgroups v2 in Alpine.
Reverting to a systemd-based distro is also being considered.

[pastukhov]

Any news?

[abiosoft]

Any news?

I am just getting some focus time for the project again. Hopefully there should be news within two weeks.