investigate why we need > 2G memory for live ISO/PXE boot

[dustymabe]

A lot of people are hitting #388. In the discussion in the documentation PR coreos/fedora-coreos-docs#46 a few of us seem to think we should be able to get by with less than 2G of RAM so we should investigate why we need more than that to see where the disconnect is.

[jlebon]

Did some more probing of this.

With 1024M of RAM, the kernel and initrd are loaded successfully, but the kernel is never started.

With 2048M of RAM, the kernel can start, but systemd crashes.

One thing I was wondering was whether somehow unzipping all the CPIOs took up all the space. Though doing a free -h at rd.break shows:

sh-5.0# /sysroot/usr/bin/free -h
              total        used        free      shared  buff/cache   available
Mem:          2.9Gi        50Mi       2.1Gi       728Mi       767Mi       2.0Gi
Swap:            0B          0B          0B

(This is before systemd nukes everything.)

[dustymabe]

I just ran some tests with the latest next release (32.20200824.1.0).

• For the live ISO I can use --ram 1024 in my virt-install command
• For PXE I seemed to need 2048 for either the concatenated initramfs/rootfs case OR or the case where I specify coreos.live.rootfs_url=

[jlebon]

I think we can close this now. 1G for the live ISO seems reasonable and 2G for PXE makes sense too since it needs to host the full rootfs.

[dustymabe]

Agreed. I just ran a test against the artifacts from 33.20201006.1.0 and was able to do a PXE boot at 2G of memory (when using coreos.live.rootfs_url like is shown in the docs) and an ISO boot at 1G of memory.

[bgilbert]

Okay, coreos/fedora-coreos-docs#196 updates docs for the current RAM requirements.

[jlebon]

Given coreos/fedora-coreos-docs#196 (comment), I think there's still a mystery re. why the concatenated approach needs more RAM. But at least the other more common cases are under control now. Let's keep this closed and we can re-open this or a separate issue to investigate what's going on in the concatenated approach if it becomes a sore point.

[bgilbert]

That behavior is expected, no? The concatenated approach should require 2x rootfs RAM during unpack, but the streaming approach should require 1x.

[jlebon]

The concatenated approach should require 2x rootfs RAM during unpack, but the streaming approach should require 1x.

Right, but shouldn't the memory taken up by the CPIOs be freed once the kernel is done extracting them? So it would need only roughly 700M x 2 + some more memory to get the CPIOs extracted. I guess this gets into the specifics of how the bootloader loads the initrds and how the kernel processes them, which changes per architecture. It's likely that how that works is just not matching my mental model and there's not much for us to improve on here. :)