Rust oak_loader binary is not reproducibly buildable (ahash::SEED)

[tiziano88]

To reproduce (no pun intended):

./scripts/build_server -s rust
cp ./target/debug/oak_loader /tmp/oak_loader_1
cargo clean
./scripts/build_server -s rust
cp ./target/debug/oak_loader /tmp/oak_loader_2
diff /tmp/oak_loader_1 /tmp/oak_loader_2
diff <(readelf --wide --all /tmp/oak_loader_1) <(readelf --wide --all /tmp/oak_loader_2) 

output of the final diff:

93670c93670
<   GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 66131c2615b7073053abf9cc7d32d375f88c1fe7
---
>   GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build ID: 6d977ef231ac71b7a70ccfc3c0e266bfd0874d33

It seems that the NT_GNU_BUILD_ID field is populated with a random string at each build invocation.

I guess we could strip that out in a post-build step, but it would be nice to figure out if there is a cargo or rustc flag to avoid this in the first place.

cc @ipetr0v

Blocks #861

[tiziano88]

Update:

#863 fixed the NT_GNU_BUILD_ID diff

There is still another diff left (following the first few steps from above, then):

diff <(xxd /tmp/oak_loader_1) <(xxd /tmp/oak_loader_2)
784391c784391
< 00bf8060: ffff ffff ffff ffff b65a 25e0 aec7 9b54  .........Z%....T
---
> 00bf8060: ffff ffff ffff ffff 82f8 65fa 38ac ab5d  ..........e.8..]

I have no idea what this is about, but it does not seem to be an ELF header: the output of readelf --wide --all is the same for both binaries.

[tiziano88]

Maybe getting to the bottom of this:

Disassembling around that section with objdump -x -D, the section that differs is this:

  0000000000bf9068 <_ZN5ahash4SEED17h75898f69630b4c3aE>:
!   bf9068:	b6 5a                	mov    $0x5a,%dh
!   bf906a:	25 e0 ae c7 9b       	and    $0x9bc7aee0,%eax
!   bf906f:	54                   	push   %rsp

which to me looks like a random seed of some sort that gets generated by the compiler and baked into the resulting binary?

[tiziano88]

Relevant section with demangled name:

0000000000bf9068 g     O .data  0000000000000008              ahash::SEED

[daviddrysdale]

BTW, c++filt is useful for this sort of thing:

% echo _ZN5ahash4SEED17h75898f69630b4c3aE | c++filt
ahash::SEED
% dpkg -S `which c++filt`
binutils: /usr/bin/c++filt

[daviddrysdale]

Probably this, which looks to be under the compile-time-rng feature of ahash

[daviddrysdale]

The NT_GNU_BUILD_ID thing doesn't seem to needed for our Bazel-built binaries – that section isn't present in the binaries.

[daviddrysdale]

The C++/Rust combined build seems to at least have "naïve reproducibility" (two consecutive clean builds give the same binary output, implying that there's no explicit randomness or timestamping in the build).