Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accepted convention for incurring dependencies on external files in macro expansion hurts reproducibility #75887

Open
pikajude opened this issue Aug 24, 2020 · 0 comments
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) A-reproducibility Area: Reproducible / deterministic builds O-NixOS Operating system: NixOS, https://nixos.org/ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@pikajude
Copy link

pikajude commented Aug 24, 2020

According to this thread https://internals.rust-lang.org/t/pre-rfc-add-a-builtin-macro-to-indicate-build-dependency-to-file/9242/2 and a few other sources I found online, the common way to force a module with a proc-macro to be rebuilt based on external changes (for example, an HTML template) is to generate an expansion that uses include_bytes!() with the path to the file.

This function in librustc_span adds the string content of every macro input to the resulting rmeta file, including macros that are then re-expanded, like this one in the askama-derive crate. That macro expansion becomes include_bytes!("/absolute/path/to/source.html");, then the actual byte literal that contains the file contents.

I am using the Nix language/build system here and the cargo2nix tool, which causes every crate to be built in a temporary directory (to maximize isolation from the rest of the system). If the tempdir names differ between two builds of a crate that otherwise logically yields identical code, the metadata hashes will differ. (Though I haven't tested it myself, I can imagine this also being a problem with a tool like sccache if two devs with different filepaths to some git checkout try to use the same build artifacts).

@jonas-schievink jonas-schievink added O-NixOS Operating system: NixOS, https://nixos.org/ A-reproducibility Area: Reproducible / deterministic builds labels Aug 24, 2020
@jieyouxu jieyouxu added A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) A-reproducibility Area: Reproducible / deterministic builds O-NixOS Operating system: NixOS, https://nixos.org/ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

3 participants