GitHub Actions: Speed up CI

[pothos]

Current situation

The CI takes quite long because it uploads a 2 GB binary package tar ball and other things needed for the kola tests and then downloads that again.

Impact

The package build speedup reduced the build time but now the bottleneck is in the up- and download.

Ideal future situation

We run the kola tests as step of the first workflow so that we can stop uploading the binpackages for the devcontainer. We can still offer it as separate workflow if wanted but then we should skip the test that needs binpackges (skipping logic can exist inside the kola test).

Edit: we can also set --image_compression_formats=none instead of bz2 because the images use internal compression and we have the zip compression on top, we don't need 3 layers.

[t-lo]

If we run the kola tests in the same build step - which makes a lot of sense to me! - we could skip uploading any artifacts other than the test results. A quick calculation from a recent run shows that we would save about 8GB of uploads, ie. 4GB per architecture.

I don't really see a need to keep offering a separate kola run; CI should be ephemeral. I get that we sometimes might just want to re-run tests, but we should explore other options for that, e.g. use Bincache.

[pothos]

Being able to download the built image is quite useful for extra testing without having to start a Jenkins or local build first.

[t-lo]

Is this something people do? For digging into test run failures? Hmmm.

Which one would we keep, the generic one or the vm-image one? IIRC the generic one is used for testing, no?

[pothos]

When reviewing PRs it's good for independent testing, for external PRs even more.

[t-lo]

Which image would we want to upload, the generic one (flatcar_production_image.bin) or the vendor image (flatcar_production_qemu_uefi_image.img)? The generic is used for running tests, right? But the vendor image is better for testing interactively.

[pothos]

Sometimes the upload is very fast: 1m 30s for the bin packages, and <30s for the VM images. I think the generic image, the UEFI image, and the PXE image are all good because they cover different use cases. The PXE image is very handy for quick tests with Ignition because one can boot it multiple times without having to clean up old state, and one can easily test out kernel parameters.

[t-lo]

The PXE image is very handy for quick tests with Ignition because one can boot it multiple times without having to clean up old state, and one can easily test out kernel parameters.

I would argue that -snapshot would offer the same for the qemu image 😉