Building a different Arch than the host machine

[rhatdan]

One of the things that came up during the hackfest was the need to build an X86 Disk image from an arm (MAC) platform.

podman build --arch=x86
podman run --arch=x86 ...
podman run --privileged bootc-image-builder --arch=x86

Is what we need, Then have bootc-image-builder run the internal skopeo or podman commands with the --arch commands.

If you are doing --arch, you should also do --variant.

[mvo5]

We did some work on this in #139

[rhatdan]

I added comment there that I think this is the wrong way. Everything runs really slowly in emulation mode, if we can run the commands in native and just pull and convert in alternative arch, the performance will be much better.

[mvo5]

I added comment there that I think this is the wrong way. Everything runs really slowly in emulation mode, if we can run the commands in native and just pull and convert in alternative arch, the performance will be much better.

Maybe I did a poor job explaining this PR but I think we are on the same page and the PR is doing what you describe. It's essentially:

1. podman runs on the native arch
2. bib/osbuild run with the native arch and downloads the target container for the native arch to construct it's buildroot (the buildroot is where osbuild finds it's tooling like mkfs, ostree etc) [this assume there is a multi-arch build which is true for e.g. {centos,fedora}-bootc]
3. there are stages that require to run "in-target" code in the deployment tree, I don't think we actually use any during a typcial bootc build but to support those qemu-user is installed so that code running in the target arch in emulation is supported. Note that this is going to be either none or very little native code
4. during the build the bootc container of the target arch is fetched and put onto the target disk

Is this making more sense now? And in line with what you have in mind?

[cgwalters]

Hmm that approach makes sense except it clashes with #18 I believe because we'd need to full-emulate bootc then - or bootc would have to more carefully separate the parts which are just e.g. writing files versus bits that are executing code.

That said while I suspect this approach may mostly work in the short term, for any kind of general case it gets hard because running e.g. mkfs.ext4 may actually use the host page size for the filesystem layout. The classic case here is ppc64le with 64k pages, which can't be read on 4k x86_64 for example.

This is also relevant here because MacOS ARM uses 16k pages on the host...and while we just use 4k on our Linux VMs, there is some motion to support 16k Linux aarch64 (driven by e.g. Asahai) and I think if that happens we'd need to still be careful to have the default filesystem block sizes be 4k in order to be readable on x86_64.

[mvo5]

Thanks Colin - I wasn't aware of those, that is slightly sad and probably means my approach is flawed. I wonder what else we can do, it may mean full emulation for the buildroot? Which still might be okay and worth benchmarking I suppose :) Or do we have more options that I don't see?

[rhatdan]

We can try full emulation, but I still stick by the fact that it will not perform well.

[cgwalters]

For more information on the filesystems and page size see https://kernelnewbies.org/KernelProjects/large-block-size

I guess in practice right now for us, the default is 4k pages so we'll create 4k block sizes, and then it'll always be "bs <= ps" which is fine. C9S for example has a kernel-64k on ARM that we should support - but it can read a 4k bs filesystem.

[mvo5]

This is merged now via #179