reproducible artifacts

[smira]

Feature Request

We need to make sure that all Talos artifacts in _out/ are reproducible.

Our initramfs-<arch>.xz is already reproducible, so we need to fix other artifacts to be reproducible as well.

Description

kernel should be fine as it gets copied directly from the image.

Disk/VM images might need some work to make sure they are reproducible: checking/resetting timestamps, making sure that order of the files in the archive is consistent.

End goal is to make sure that:

1. Build all artifacts.
2. Capture sha256 sums of the file in _out/.
3. Clear _out, drop buildkit caches.
4. Rebuild all artifacts.
5. Capture sha256 again, it should match checksumes from step 2.

[smira]

current status:

• vmlinuz-*, initramfs-*, talosctl-* are reproducible
• installer and talos images are reproducible
• *.iso images are reproducible
• disk images of various kinds are not reproducible as they contain xfs filesystems which in turn contain creation time and that depends on the host time; as the installer image is reproducible though, any disk image can be generated with it

[flokli]

disk images of various kinds are not reproducible as they contain xfs filesystems which in turn contain creation time and that depends on the host time; as the installer image is reproducible though, any disk image can be generated with it

Could all the partitions that need to exist in the image itself be ext4 (or squashfs), and you only create xfs during apply-config? That should keep the images themselves binary reproducible, if you use make_ext4fs or mksquashfs with some patches.

It seems there's not really something for xfs yet.

[smira]

It might be even not xfs, but also VFAT for boot partitions. We don't see that being the problem, as the installer and imager containers are reproducible.

[flokli]

I recommend using diffoscope to track these down. See https://r13y.com/

[smira]

thanks, we are already using that: https://github.com/talos-systems/tools/blob/master/Makefile#L38-L46

[smira]

with the last pieces of work on bldr and package repositories, I think we can close this issue:

• feat: implement SOURCE_DATE_EPOCH build argument bldr#83
• feat: use bldr with support for SOURCE_DATE_EPOCH toolchain#33
• chore: reproducible builds tools#172
• chore: reproducible builds pkgs#412
• chore: reproducible builds, use pkgs base extras#41