Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework initrd usr mount to start Ignition/Afterburn from there #52

Merged
merged 3 commits into from
Jan 25, 2023

Conversation

pothos
Copy link
Member

@pothos pothos commented Jan 20, 2023

  • Rework initrd usr mount to start Ignition/Afterburn from there

    The size of the unified kernel image grew so much that we almost hit
    the limit of the fixed /boot partition. While we could tweak which
    kernel modules are present the largest contributors are actually
    Ignition and Afterburn (coreos-metadata). Since we also ship Afterburn
    on the /usr partition and have Ignition installed there, too (but
    currently masked), we could do what was done for Torcx and directly
    call it from there (but with LD_LIBRARY_PATH instead of chroot). There
    are a few corner cases where this was not directly possible due to the
    order of events in the initrd and these had to be reworked.

    This establishes a structure that allows the /sysusr/usr mount to be
    used for calling Ignition and Afterburn. This mount point was set up
    by systemd already but since we have a legacy generator, this needed
    adaption, too. The final /usr mount in /sysroot/usr is still set up.
    In the case of the systemd generator it is a bind mount, for the
    legacy generator it is a separate mount, both work but we could align
    this to also use a bind mount. The afterburn-network-kargs.service
    starts a bit later now because it depends on the /sysusr mount and
    this also means that the parse-ip-for-networkd dracut hook needed to
    run at a later stage, here solved with an own service but since the
    dracut hook mechanism with sourcing is complicated, the script still
    runs as hook to serialize the environment. The fsck check for /usr was
    also in the way and thus we write out a dummy systemd-fsck-usr.service
    since we don't need to check /usr as it's verity-protected. In the end
    with these changes we significantly reduce the unified kernel image
    size and have more breathing room for the next years. If we have to
    move some other binaries or even kernel modules that are not essential
    for loading /usr itself, we can move them out of the initrd based on
    this work. Size reduction for arm64 is 58 MB -> 52 MB, for amd64 it is
    55 MB -> 49MB.

  • Align check for initrd-provided /usr

  • Fix non-working norecovery option
    The check for the norecovery option used the same condition but had
    different actions in the if branch. Also, for the systemd generator
    the unit name was wrong because it was just the unit name of the bind
    mount.
    Align the check for the norecovery option and also use the right unit
    name.

How to use

Testing done

CI run

Tested the usr= cmdline case manually, tested the first boot ip=10.0.2.15::10.0.2.2:255.255.255.0::eth0:off:: cmdline case (and once with netroot=dummy) and ran the CI to test the kargs VMware case and that Ignition works.

@pothos pothos requested a review from a team January 20, 2023 15:39
pothos added a commit to flatcar-archive/coreos-overlay that referenced this pull request Jan 20, 2023
This pulls in
flatcar/bootengine#52
to reduce the size of the unified kernel image by starting Ignition/
Afterburn from the /usr partition.
This also requires to install the Ignition binary to /usr.
dracut/03flatcar-network/parse-ip-for-networkd.sh Outdated Show resolved Hide resolved
dracut/03flatcar-network/parse-ip-for-networkd.sh Outdated Show resolved Hide resolved
dracut/10usr-generator/usr-generator Outdated Show resolved Hide resolved
dracut/10usr-generator/usr-generator Outdated Show resolved Hide resolved
dracut/10usr-generator/usr-generator Show resolved Hide resolved
The check for the norecovery option used the same condition but had
different actions in the if branch. Also, for the systemd generator
the unit name was wrong because it was just the unit name of the bind
mount.
Align the check for the norecovery option and also use the right unit
name.
The size of the unified kernel image grew so much that we almost hit
the limit of the fixed /boot partition. While we could tweak which
kernel modules are present the largest contributors are actually
Ignition and Afterburn (coreos-metadata). Since we also ship Afterburn
on the /usr partition and have Ignition installed there, too (but
currently masked), we could do what was done for Torcx and directly
call it from there (but with LD_LIBRARY_PATH instead of chroot). There
are a few corner cases where this was not directly possible due to the
order of events in the initrd and these had to be reworked.

This establishes a structure that allows the /sysusr/usr mount to be
used for calling Ignition and Afterburn. This mount point was set up
by systemd already but since we have a legacy generator, this needed
adaption, too. The final /usr mount in /sysroot/usr is still set up.
In the case of the systemd generator it is a bind mount, for the
legacy generator it is a separate mount, both work but we could align
this to also use a bind mount. The afterburn-network-kargs.service
starts a bit later now because it depends on the /sysusr mount and
this also means that the parse-ip-for-networkd dracut hook needed to
run at a later stage, here solved with an own service but since the
dracut hook mechanism with sourcing is complicated, the script still
runs as hook to serialize the environment. The fsck check for /usr was
also in the way and thus we write out a dummy systemd-fsck-usr.service
since we don't need to check /usr as it's verity-protected. In the end
with these changes we significantly reduce the unified kernel image
size and have more breathing room for the next years. If we have to
move some other binaries or even kernel modules that are not essential
for loading /usr itself, we can move them out of the initrd based on
this work. Size reduction for arm64 is 58 MB -> 52 MB, for amd64 it is
55 MB -> 49MB.
@pothos pothos merged commit aa8a19e into flatcar-master Jan 25, 2023
@pothos pothos deleted the kai/initrd-rework branch January 25, 2023 16:01
pothos added a commit to flatcar-archive/coreos-overlay that referenced this pull request Jan 25, 2023
This pulls in
flatcar/bootengine#52
to reduce the size of the unified kernel image by starting Ignition/
Afterburn from the /usr partition.
This also requires to install the Ignition binary to /usr.
t-lo pushed a commit to flatcar/scripts that referenced this pull request Apr 17, 2023
This pulls in
flatcar/bootengine#52
to reduce the size of the unified kernel image by starting Ignition/
Afterburn from the /usr partition.
This also requires to install the Ignition binary to /usr.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants