proof of concept for buildx in github actions

[allisonkarlitskaya]

this works, but wow is it slow

the biggest problem is dnf being so slow. if we could figure out a way to get microdnf in there, it might help things, but installing it would use... dnf. either that, or figure out a way to do the depsolving on x86 and install the result based on that...

i guess we can't rely on having the exact same package list across both platforms...

i don't think this is really workable in its current state, unfortunately

[allisonkarlitskaya]

Here's the test run: https://github.com/allisonkarlitskaya/cockpituous/actions/runs/8288718419/job/22683794778

It took over an hour to run... the log format of the docker action is really helpful: you can use the left column (#10) to grep apart lines from parallel tasks, and the next column is the relative number of seconds since the start of the run, which is useful for identifying which specific steps took longest (even down to which individual package postinsts are slow — like the selinux ones — which gives pointers for optimization possibilities).

I found this: https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/ (and filled in the form).

This is also a thing, and they have native arm runners: https://buildjet.com/for-github-actions. I tried out a simple arm-only build here: https://github.com/allisonkarlitskaya/cockpituous/actions/runs/8289214204/job/22685231937 (workflow: https://github.com/allisonkarlitskaya/cockpituous/blob/buildjet/.github/workflows/build-tasks.yml). That was a lot faster (~4 minutes, at a cost of 2 cents), and the setup was really smooth — it worked on the first try. We might imagine having two build jobs, one on arm, one on x86, and then a job which runs at the end to create and upload the manifest... because runs-on is per-job, we'd have the possibility to send only the arm job over to them.

I also tried doing this on my raspberry pi and dnf pinned the CPU at 100% for a long time, but (after changing the storage driver from vfs to overlay) the overall performance wasn't terrible. I'm not sure we want to rely on something like that, though...

[martinpitt]

Thanks for the experiments! It seems very reasonable to pay some money for building them properly TBH -- the issue is not at all the 20 cents that we're going to spend each year, but the bureaucracy around it. Although at this price range we can keep this really simple (I'm happy to just pay it for myself, too).

Does this work as two independent build jobs? i.e. if you upload a built container image to the registry, do you have to upload all arches at the same time, or can you publish them individually? I seriously hope the latter, as it seems to be rather impossible to build 5 arches (like distributions do) on the same place

[allisonkarlitskaya]

Does this work as two independent build jobs? i.e. if you upload a built container image to the registry, do you have to upload all arches at the same time, or can you publish them individually? I seriously hope the latter, as it seems to be rather impossible to build 5 arches (like distributions do) on the same place

You create two images with different named tags (like 2024-03-15-arm or so) and upload them. Then you create a manifest under the "public" name which refers to the separate images. I didn't try it let but I'm sure I can get it working today...