What is your deployment path? bootc-image-builder? Anaconda? bootc install to-filesystem wrapped with something else?

Is this after an in-place update or is this a fresh install issue?

I've seen it both ways.

In bios mode today, I did an upgrade with the new image and it seemed to work. then reinstalled forcing the firmware over to efi. it installed without the args. Then I tweaked the image then in-place updated it and still no options applied.

The thing is with the default bootloader flow we have, bios and UEFI both invoke grub which both just read the bootloader configs from /boot so there should really be no possible difference between the two.

Can you please include the container OS version and confirm the bootloader path?

FROM quay.io/almalinuxorg/almalinux-bootc:9.4-20241023
RUN echo 'kargs = ["ipv6.disable=1"]' > /usr/lib/bootc/kargs.d/00-config.toml

When I boot, I see the config.toml

# cat  /usr/lib/bootc/kargs.d/00-config.toml
kargs = ["ipv6.disable=1"]

But the generated loader args are skipping the kargs.d file:

# more /boot/loader.1/entries/ostree-1.conf 
title AlmaLinux 9.20241023.0.4 (Seafoam Ocelot) (ostree:1)
version 1
options crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M root=UUID=43eb3e40-ba57-4699-a52b-97c31139cf1f rw ostree=/ostree/boot.1/default/cbeefe89fd291c49b81b228f857a42b65366392f437dbd0489b160b1cbb50dab/1
linux /ostree/default-cbeefe89fd291c49b81b228f857a42b65366392f437dbd0489b160b1cbb50dab/vmlinuz-5.14.0-427.40.1.el9_4.x86_64
initrd /ostree/default-cbeefe89fd291c49b81b228f857a42b65366392f437dbd0489b160b1cbb50dab/initramfs-5.14.0-427.40.1.el9_4.x86_64.img
aboot /ostree/deploy/default/deploy/bc3560ecaf50123a8d79b2dcdf73b74df1ec274b6c98a164a93c9196671c2b35.0/usr/lib/ostree-boot/aboot.img
abootcfg /ostree/deploy/default/deploy/bc3560ecaf50123a8d79b2dcdf73b74df1ec274b6c98a164a93c9196671c2b35.0/usr/lib/ostree-boot/aboot.cfg

We chatted in person about this and realized it's because Anaconda isn't using bootc install yet, but we're working on that!

So, fixing Anaconda won't be quick I think....

What workaround might we use until then? bootc upgrading didn't seem to apply the kernel args either. Is there some way we could get it to reset things so that bootc upgrade after install would apply the container kargs?

The most viable hack would be in %post but it'd be pretty ugly, we're actively working on switching anaconda to run through the bootc flow.

With RHEL 9.5 just being released though, it will probably be a long time before a new anaconda drops.

I'm curious what the workaround might be. Maybe just a rough idea on whats going on. Even bootc upgrade --apply doesn't apply the new image kargs. Does machine kargs (via kickstart) somehow have precedence over the container ones and prevents them from being applied? If so, how might those be looked at/reset?

I'm curious what the workaround might be.

I will do some investigation if we can do something like this:

%post --erroronfail
bootc install finish
%end

or so...this would allow us to avoid needing to respin ISOs, but it'd likely be somewhat hacky..

Even bootc upgrade --apply doesn't apply the new image kargs.

Yes, because we diff between the old and new state and we expect the old state to be valid.

So we have active investigation right now for handling automatically making Anaconda installs Do The Right Thing, then after that works we'll investigate something like this %post bootc install finish.

Thanks for the insight! That helped narrow things down. I was able to boot an image that didnt have any overrides, then bootc switch image to one with custom kargs in the image, and have that work. So I have a not horrible workaround for now.

PR in #915