Initial commit of a Debian sid image for toolbox

[bureado]

This is inspired by @kronenpj work in #298 but for Debian sid, so a few changes here and there.

It builds with podman and no issues with toolbox create or toolbox enter on a Debian sid host.

[softwarefactory-project-zuul]

Build succeeded.

• shellcheck : SUCCESS in 1m 30s
• test-with-podman-stable-f30 : SUCCESS in 8m 18s
• test-with-podman-stable-f31 : SUCCESS in 9m 33s
• test-with-podman-rawhide : SUCCESS in 17m 31s

[martinpitt]

Just yesterday I had a similar idea ("use toolbox for my Debian development"), and came up with a little script to set this up.

You don't install libnss-myhostname. Without that, sudo (and other tools) complain that they can't resolve the toolbox host name. Did you somehow not run into that?

Also, note to myself: toolbox in Fedora uses #!/usr/bin/sh, which is a bit awkward as that does not exist in Debian. But this isn't an upstream issue, there it uses the standard #!/bin/sh. Apparently Fedora package builds change it during install somewhere.

[martinpitt]

FTR, I now improved my script to be able to build a debian or ubuntu container for any release from the stock dockerhub images.

[philn]

@martinpitt I've patched my toolbox as in #401, downloaded your build-debian-toolbox script, but it doesn't work here:

build-debian-toolbox                                                                                                                                                                                  18:28:07
+ RELEASE=sid
+ DISTRO=debian
+ toolbox rm -f sid
toolbox: failed to inspect sid
+ true
+ toolbox -y create -c sid --image docker.io/debian:sid
Created container: sid
Enter with: toolbox enter --container sid
+ podman start sid
sid
+ podman exec -it sid sh -exc '
# https://bugzilla.redhat.com/show_bug.cgi?id=1785244, https://github.com/containers/toolbox/pull/380
if [ ! -e /etc/resolv.conf ]; then
    ln -sfn /run/host/monitor/resolv.conf /etc/resolv.conf
fi

# go-faster apt/dpkg
echo force-unsafe-io > /etc/dpkg/dpkg.cfg.d/unsafe-io

# location based redirector gets it wrong with company VPN
# sed -i "s/deb.debian.org/ftp.de.debian.org/" /etc/apt/sources.list

apt-get update
apt-get install -y libnss-myhostname sudo eatmydata libcap2-bin

# allow sudo with empty password
sed -i "s/nullok_secure/nullok/" /etc/pam.d/common-auth
'
Error: can only create exec sessions on running containers: container state improper

[philn]

Ah it seems this might be due to dangling images in podman, from previously failed attempts. I managed to get a working focal toolbox now. :)

[philn]

Well actually, all I can create is a focal toolbox... :( Even after a podman system reset

[martinpitt]

@philn: I created a debian testing toolbox successfully just yesterday.. I just tried build-debian-toolbox bionic ubuntu and it works (at least it's installing debs now, and I can shell in). What precisely fails for you now?

[philn]

The error pasted above, Error: can only create exec sessions on running containers: container state improper... I'll try to debug this then.

[philn]

Also, note to myself: toolbox in Fedora uses #!/usr/bin/sh, which is a bit awkward as that does not exist in Debian. But this isn't an upstream issue, there it uses the standard #!/bin/sh. Apparently Fedora package builds change it during install somewhere.

That was the problem, changing my local custom toolbox script to use the bash shebang fixed the issue.

[heyakyra]

What remaining blockers are there on this?

[biji]

Successfully create ubuntu-toolbox 18.04 using build-debian-toolbox and run bzr explorer

[j1mc]

Bumps : )

[martinpitt]

I'm not a toolbox project maintainer, but honestly this whole "toolbox specific images" approach seems flawed to me. It has not worked very well even for Fedora (e.g. Fedora 34 images on https://registry.fedoraproject.org/repo/fedora-toolbox/tags/ are almost a month old!) and I don't see this getting any better with Debian, Ubuntu, and more OSes.

To me it seems a much better approach is make/keep toolbox working with the official distro images, and put that into CI, rather than trying to maintain an entirely parallel set of toolbox specific images.

I have built toolboxes from official Debian, Ubuntu, and cockpit/tasks images for a long time without any problem, and it works so much better. Sure it needs a small hack to bootstrap for Debian/Ubuntu, but it Just Works for Fedora images (my script).

[martinpitt]

@debarshiray ^ Any opinion about the strategic direction here? (See previous comment)

[debarshiray]

The fedora-toolbox images are a bunch of packages on top of the fedora images. So, it's entirely possible to script them locally.

(Sidenote: the definition for the fedora-toolbox images are part of toolbox.git. So one could also do a podman build ... locally against those.)

The fedora-toolbox images exist because the entry point of the containers, which is /usr/bin/toolbox itself, needs a few binaries to be present inside the container, and those might not be present in the base fedora image. eg., passwd(1).

Other than that, the fedora-toolbox images try to match the CLI user experience with that of a Fedora Workstation or Silverblue host. The base images are optimized for size, and typical non-interactive container use. They don't have manuals, in the past they came with coreutils-single not coreutils, plus a bunch of other tools are missing from them.

We could pull in these packages at run-time, but that will be at the cost of giving up the determinism that an image provides. Everybody sees the same image on the registry, and we aren't exposed to some random RPM transaction going wrong.

And RPM transactions do go wrong, not just because of the usual problems of packages, but also because of the inherent nature of rootless OCI containers. eg., there's a group of packages that are picky about the file ownerships of some locations like /proc, /dev/, etc., and these locations can be owned by nobody:nobody if bound mounts from the host are in place. I believe the most pesky is the filesystem RPM.

So, it's a trade-off. I am open to shifting the balance, if we can work through these issues.

[martinpitt]

@debarshiray Right, it's a trade-off. But I think the only other reasonable option is to add a bunch of workflows/pipelines that build, validate, and push all supported OS containers every day. Involving human reporters to find and report issues like #722 and then involving human developers to refresh them doesn't appear that great to me for anyone involved?

[debarshiray]

About the age of the fedora-toolbox images, I am not so worried about the one month part. We can definitely re-spin them more frequently. It's usually @juhp who does them, and is a matter of running fedpkg container-build and then getting them through Bodhi. The image definitions don't change so often.

Now that we have support for RHEL containers through the UBI images, @HarryMichal and I have been reaching out to various people who are interested in other distributions. Every once in a while, we run into an issue concerning the distro itself, or a Toolbox issue that affects a specific distro; extending and maintaining the CI, etc..

Ubuntu is by far the most popular demand, so we started with that. Unfortunately, people tend to balk at any long-term maintenance, no matter how small, as opposed to the thrill of submitting a one-off pull request. :)

[heyakyra]

a small hack to bootstrap for Debian/Ubuntu

Brilliant, this is at least something people can use in the meantime. Thank you!

[travier]

We've made progress in this front in #1019. See the top comment for links to community supported images.

[juhp]

Can this issue be closed now?

https://github.com/toolbx-images/images/tree/main/debian
and quay.io/toolbx-images/debian-toolbox