Add 40coreos dracut module

[jlebon]

This module is the integration point between Ignition and Fedora CoreOS.
Notably, we add two new systemd services in the initrd which mesh with
Ignition's own systemd services:

1. coreos-mount-var.service, which finds the /var stateroot bind
   mount and mounts it under /sysroot/var before ignition-mount
   potentially adds more mounts underneath (or even shadowing /var
   itself).
2. coreos-populate-var.service, which sets up /var on first boot
   using systemd-tmpfiles before ignition-files runs.

[cgwalters]

How would we handle /var as a real partition?

[jlebon]

How would we handle /var as a real partition?

The mount stage will mount it just like any other. The bind-mount we do here is for the case where only e.g. /var/home is specified, in which case, we still want to populate all the other stuff under /var that would go in the stateroot. Instead of doing it conditionally, we just always bind-mount and always unmount to keep it simple (that's the "shadowing" part in the commit message).

Re. /var in FCOS specifically, I think the current idea is to ship that as a base config that users can override/tweak as they want.

[dustymabe]

so I know we are wanting ignition-dracut repo to be mostly distro agnostic but would there be harm in having code in there that activates on one distro but is dormant on another? i.e. it would be kind of nice to just put this module in ignition-dracut and then non-ostree based distros could leave it out.

cc @ajeddeloh @bgilbert for input. It's also possible there was another conversations that I missed or forgot that led us here to begin with.

[jlebon]

OK, this is working pretty well. Two things that bit me were https://src.fedoraproject.org/rpms/ignition/pull-request/31 and coreos/ignition#771. I played with various mounts, e.g. /var, /var/lib/containers, /var/home, as well as all three at the same time!

[root@localhost ~]# findmnt | grep /var
├─/var                    /dev/vdc1  xfs  rw,relatime,seclabel,attr2,inode64,noquota
│ ├─/var/home             /dev/vdb1  xfs  rw,relatime,seclabel,attr2,inode64,noquota
│ └─/var/lib/containers   /dev/vdd1  xfs  rw,relatime,seclabel,attr2,inode64,noquota

For anyone who wants to follow along, how I'm testing this right now is with:

diff --git a/fedora-coreos-base.yaml b/fedora-coreos-base.yaml
index fef318a..95a732a 100644
--- a/fedora-coreos-base.yaml
+++ b/fedora-coreos-base.yaml
@@ -58,6 +58,11 @@ remove-from-packages:
   # NOTE: Also remove 01_fallback_counting once we move to f30
   - [grub2-tools, /etc/grub.d/01_menu_auto_hide,
                   /usr/lib/systemd/.*]
+  # XXX
+  - [ignition, /usr/lib/dracut/modules.d/30ignition]
+
+add-files:
+  - [ignition, /usr/lib/dracut/modules.d/31ignition/ignition]


 # ⚠⚠⚠ ONLY TEMPORARY HACKS ALLOWED HERE; ALL ENTRIES NEED TRACKER LINKS ⚠⚠⚠

And rsync -al of 30ignition in coreos/ignition-dracut#47 at overlay/usr/lib/dracut/modules.d/31ignition.

(I was hitting issues packaging ignition git master with the recent git module rework, so just went with the nuclear option.)

And then a VM with an extra disk added and a config like:

    "storage": {
        "disks": [
            {
                "device": "/dev/vdb",
                "wipeTable": true,
                "partitions": [
                    {
                        "label": "VAR",
                        "number": 0,
                        "sizeMiB": 0,
                        "wipePartitionEntry": true
                    }
                ]
            }
        ],
        "filesystems": [
            {
                "device": "/dev/vdb1",
                "path": "/var",
                "format": "xfs",
                "wipeFilesystem": true,
                "label": "VAR"
            }
        ],
        "files": [
            {
                "path": "/etc/fstab",
                "append": [
                    {
                        "source": "data:,%0A/dev/vdb1%20/var%20xfs%20defaults%200%200%0A"
                    }
                ]
            },
            {
                "path": "/var/lib/hello.txt",
                "contents": {
                    "source": "data:,hello%20world"
                }
            }
        ]
    }

[jlebon]

I dropped WIP on this, but do note that this requires coreos/ignition-dracut#47, which is pending on FCOS switching to the new Ignition spec. So we need to hold this for now. That said, it's ready for review, and it'd be good if someone else tested this as well!

[ajeddeloh]

LGTM

[jlebon]

i.e. it would be kind of nice to just put this module in ignition-dracut and then non-ostree based distros could leave it out

Here's a question related to this: let's say we keep them separate, then couldn't we just fold the Ignition dracut module into https://github.com/coreos/ignition? Other than ignition-setup, all the services there are solely about integrating Ignition into an initrd using stable systemd targets. We could probably move off the few other stuff remaining and drop the ignition-dracut repo entirely.

[dustymabe]

i.e. it would be kind of nice to just put this module in ignition-dracut and then non-ostree based distros could leave it out

Here's a question related to this: let's say we keep them separate, then couldn't we just fold the Ignition dracut module into https://github.com/coreos/ignition? Other than ignition-setup, all the services there are solely about integrating Ignition into an initrd using stable systemd targets. We could probably move off the few other stuff remaining and drop the ignition-dracut repo entirely.

Perhaps we could move ignition-dracut into ignition but probably not worth it right now. I'm cool with this as is and we can re-org in the future if it's an attractive option.

[bgilbert]

I'm fine having this in fedora-coreos-config for now. If we put distro-specific code in ignition-dracut, everyone will want to put distro-specific code in ignition-dracut. 😄

[bgilbert]

As a starting point, won't we want a systemd-sysusers invocation in this script?

[jlebon]

I was playing with this, but I think for now we should wait for the full sysusers switchover. The issue is that it'd require fooling systemd-sysusers into somehow being aware of the entries in /sysroot/usr/lib/{passwd,group} so that it notices entries that already exist. Alternatively, we can drop from group/passwd in this repo the entries that we expect to be created by sysusers, but that seems error-prone.

The other issue is that we'll then have some sysusers in /etc/passwd and some others in /usr/lib/passwd, and I'm not sure how well rpm-ostree layering will play with that. See e.g. https://github.com/projectatomic/rpm-ostree/blob/6b2ac5885fd1c4bdff2de90a2234d03c839f76ef/src/libpriv/rpmostree-passwd-util.c#L1183.

I think we could get this to work, but I'm not sure it's worth the investment if we'll just switch to sysusers anyway.

[jlebon]

OK, this is now ready for review again!
Needs to be merged together with coreos/ignition-dracut#76.

Also requires: #85

[jlebon]

Rebased! ⬆️

[ajeddeloh]

LGTM