Use of tempfile for lib.def leads to non-reproducible PDBs

[nikhilm]

I've been working with a rust code base that we are trying to use a shared cache for (Bazel FWIW). As part of that I've been looking at truly bit-identical build outputs for a given set of inputs.

There are plenty of problems with PDB files on Windows (which contain debug info) because they tend to encode absolute paths to the object files and other resources they were created from. I've managed to eliminate most of these by using deterministic build directories across all machines, and using tempdir's that are a hash of something unique. There is just one more bit that I can't control directly, which is this use of tmpdir.join("lib.def")

rust/src/librustc_codegen_ssa/back/linker.rs

Line 720 in 0862458

let path = tmpdir.join("lib.def");

. This ends up putting a few random characters in the path that throws off the PDB.

I'm not entirely sure what a good solution will look like. I'm not familiar enough with compiler internals to know if some deterministic location based on some unique values can be used, or alternatively if this can be exposed as a compiler option.

I tried this code:
I can't share our internal code, but build something like serde_derive on windows and look for the proc macro dll. there should be a pdb next to it.
Next, you'll want to use ducible https://github.com/jasonwhite/ducible/ to remove a few other sources of non-determinism like timestamps.

ducible <dll> <pdb>

I expected to see this happen: Each clean build results in the same DLL and PDB (sha256 or similar).

Instead, this happened: the DLL is identical but the PDBs are not. Use something like https://www.cjmweb.net/vbindiff/ to spot the differences and the only one i noticed was this path to lib.def.

Meta

(also exists in latest master based on code inspection).
rustc --version --verbose:

rustc 1.42.0-nightly (212b2c7da 2020-01-30)
binary: rustc
commit-hash: 212b2c7da87f3086af535b33a9ca6b5242f2d5a7
commit-date: 2020-01-30
host: i686-pc-windows-msvc
release: 1.42.0-nightly
LLVM version: 9.0

[wesleywiser]

Visited during wg-debugging triage. A number of other reproducibility issues are being looked at currently. Hopefully we can fix this soon. Marking P-medium.