-
-
Notifications
You must be signed in to change notification settings - Fork 185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kernel is not reproducible #734
Comments
Still at it! |
Converting a bzImage to an ELF image via https://github.com/marin-m/vmlinux-to-elf worked. diffoscope takes almost an hour on the result--but it worked. Seems that the main difference is that the start of the program headers moved--everything else seems okay enough (changes a lot of address references in instructions though). The reason is a size change in the section │ - Start of program headers: 14573310 (bytes into file)
See also https://lwn.net/Articles/531148/ for the meaning of
I'd try to use the same version of elfutils on both build hosts (that's used for the "linux" module) (check which one is used), maybe that fixes it (I doubt it). elfutils is used by Linux for CONFIG_UNWINDER_ORC for generating ORC metadata. (even though the ORC sections, respectivey, are entirely equal, maybe there's still some init data generated from it) |
Those last bytes are two magic numbers for compressed files, the first one is for GZIP, and the second for XZ (see Wikipedia). It is a buffer used by function populate_rootfs, and as far as I'm aware it is built from files in /usr. The format of rootfs file should be controllable with CONFIG_INITRAMFS_COMPRESSION, which I do not know how it chooses the compression method if the option itself is disabled (I assume .config is identical in both cases, but I might be wrong about that), but then again I looked primarily on the newer sources and this part of configuration was changed couple of times. It seems that back in 4.14 it was impossible to use uncompressed rootfs. |
@krystian-hebel Thank you so muh for your analysis. If one does, for the same of commit id 9719510 for which rom was given in previous comment:
Then:
Both roms were produce with the same linux config file |
@krystian-hebel
Then:
Where |
Same applies to all other linux configs for other boards it seems (tested for linux-x230.config also) Time to explicitate that and retry builds. |
…attempt to resolve issue linuxboot#734 (thanks to @krystian-hebel)
…attempt to resolve issue linuxboot#734 (thanks to @krystian-hebel)
…attempt to resolve issue linuxboot#734 (thanks to @krystian-hebel)
@krystian-hebel other ideas? Implementing the above changes still produce non-reproducible kernels as shown under #850 (comment) |
Issue remains the same, i.e. GZIP vs XZ. |
By manually dd-ing that initrd out I can confirm that CPIOs are identical in both cases:
All differences in code seem to be just in memory addresses of data and called functions. |
…: expend linux's defconfig stored config under config/linux-qemu.config to test linuxboot#734 (comment) per @krystian-hebel comment (RD_XZ and all other known config params specify XZ as of now. Will be more easy to troubleshoot and less long to have artifacts.
@krystian-hebel Heads store configs (coreboot, linux) as So from tlaurion@653582e commit on, this PR builds only for qemu-coreboot to fasten builds. We know cpio is reproducible, and from past tests, we know that only the kernel is not reproducible as of now (review hashes.txt and compare.) The question is still why bzImage is not reproducible, if musl-cross-make is used to build the kernel and is supposed to be the same, building the same config from that musl-cross-make toolchain. On that matter, you can see tlaurion@653582e#diff-883dee0e42d0c32ec557a7bd837e6b67 what is "supposed" to be part of the
To produce such config, what was done was:
Nothing else on that matter.
|
As of CircleCI build artifact's hashes which matches even previous CI builds (normal)
|
For GitlabCI it also matches the previous build. Even with full, explicit config it seems to either a) choose a different path when parsing Makefiles, or b) re-parse configuration from Kconfig files on final |
@osresearch @BlackMaria @MrChromebox @flammit @daym
First step accomplished:
Each commit now triggers two CIs. For example: Click on the green check, and it will show you CircleCI and GitlabCI clickable builds.
Make vmlinuz reproducible again.
795b307945343c5c7eb0b55a9285bb73f96832d295da9be5e11887db804ebe02 /root/project/build/x230-hotp-verification/bzImage
69961890c7ee425855f8620562b5627584f7b2c5f52132291836445087cc90ef /builds/tlaurion/heads-osresearch/build/x230-hotp-verification/bzImage
Have heads.cpio reproducible
Have modules.cpio reproducible
Have tools.cpio reproducible
Have initrd.cpio.xz reproducible
have reproducible roms
Originally posted by @tlaurion in #571 (comment)
The text was updated successfully, but these errors were encountered: