-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a simple reproducibility test command. #13689
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
## Simple tool to test build reproducibility | ||
|
||
Meson now ships with a command for testing whether your project can be | ||
[built reprodicibly](https://reproducible-builds.org/). It can be used | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. typo: s/reprodicibly/reproducibly |
||
by running a command like the following in the source root of your | ||
project: | ||
|
||
meson reprotest --intermediaries -- --buildtype=debugoptimized | ||
|
||
All command line options after the `--` are passed to the build | ||
invocations directly. | ||
|
||
This tool is not meant to be exhaustive, but instead easy and | ||
convenient to run. It will detect some but definitely not all | ||
reproducibility issues. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
# SPDX-License-Identifier: Apache-2.0 | ||
# Copyright 2024 The Meson development team | ||
|
||
from __future__ import annotations | ||
|
||
import sys, os, subprocess, shutil | ||
import pathlib | ||
import typing as T | ||
|
||
if T.TYPE_CHECKING: | ||
import argparse | ||
|
||
from ..mesonlib import get_meson_command | ||
|
||
# Note: when adding arguments, please also add them to the completion | ||
# scripts in $MESONSRC/data/shell-completions/ | ||
Comment on lines
+15
to
+16
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Have I missed these? |
||
def add_arguments(parser: 'argparse.ArgumentParser') -> None: | ||
parser.add_argument('--intermediaries', | ||
default=False, | ||
action='store_true', | ||
help='Check intermediate files.') | ||
parser.add_argument('mesonargs', nargs='*', | ||
help='Arguments to pass to "meson setup".') | ||
|
||
IGNORE_PATTERNS = ('.ninja_log', | ||
'.ninja_deps', | ||
'meson-private', | ||
'meson-logs', | ||
'meson-info', | ||
) | ||
|
||
INTERMEDIATE_EXTENSIONS = ('.gch', | ||
'.pch', | ||
'.o', | ||
'.obj', | ||
'.class', | ||
) | ||
|
||
class ReproTester: | ||
def __init__(self, options: T.Any): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A Protocol can get rid of the need for if T.TYPE_CHECKING:
from typing_extensions import Protocol
class Arguments(Protocol):
intermediaries: bool
mesonargs: T.List[str] There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Eventually I'd like to get to the point that we don't have |
||
self.args = options.mesonargs | ||
self.meson = get_meson_command()[:] | ||
self.builddir = pathlib.Path('buildrepro') | ||
self.storagedir = pathlib.Path('buildrepro.1st') | ||
self.issues: T.List[str] = [] | ||
self.check_intermediaries = options.intermediaries | ||
|
||
def run(self) -> int: | ||
if not pathlib.Path('meson.build').is_file(): | ||
sys.exit('This command needs to be run at your project source root.') | ||
self.check_ccache() | ||
self.cleanup() | ||
self.build() | ||
self.check_output() | ||
self.print_results() | ||
if not self.issues: | ||
self.cleanup() | ||
return len(self.issues) | ||
|
||
def check_ccache(self) -> None: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Detecting all ways ccache could be injected is tricky, especially since actual compiler detection code is elsewhere and not easily callable. I guess we could check if There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Simplest approach is to just use $CCACHE_DISABLE in the environment to tell ccache that even when run, it should simply forward directly to the compiler without performing any actions itself. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This would also mean we could simply support the ccache users, as their testing would work by telling ccache to bypass the actual cache, instead of just refusing to perform the reproducibility testing at all. |
||
for evar in ('CC', 'CXX'): | ||
evalue = os.environ.get(evar, '') | ||
if 'ccache' in evalue: | ||
print(f'Environment variable {evar} set to value "{evalue}".') | ||
print('This implies using a compiler cache, which is incompatible with reproducible builds.') | ||
sys.exit(1) | ||
Comment on lines
+61
to
+66
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This doesn't take into account the approach of linking Maybe a a better solution is to set |
||
|
||
def cleanup(self) -> None: | ||
if self.builddir.exists(): | ||
shutil.rmtree(self.builddir) | ||
if self.storagedir.exists(): | ||
shutil.rmtree(self.storagedir) | ||
|
||
def build(self) -> None: | ||
setup_command: T.Sequence[T.Union[pathlib.Path, str]] = self.meson + ['setup', self.builddir] + self.args | ||
build_command: T.Sequence[T.Union[pathlib.Path, str]] = self.meson + ['compile', '-C', self.builddir] | ||
subprocess.check_call(setup_command) | ||
subprocess.check_call(build_command) | ||
self.builddir.rename(self.storagedir) | ||
subprocess.check_call(setup_command) | ||
subprocess.check_call(build_command) | ||
Comment on lines
+77
to
+81
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does it make sense to catch the exceptions that |
||
|
||
def ignore_file(self, fstr: str) -> bool: | ||
for p in IGNORE_PATTERNS: | ||
if p in fstr: | ||
return True | ||
if not self.check_intermediaries: | ||
if fstr.endswith(INTERMEDIATE_EXTENSIONS): | ||
return True | ||
return False | ||
|
||
def check_contents(self, fromdir: str, todir: str, check_contents: bool) -> None: | ||
import filecmp | ||
frompath = fromdir + '/' | ||
topath = todir + '/' | ||
for fromfile in pathlib.Path(fromdir).glob('**/*'): | ||
if not fromfile.is_file(): | ||
continue | ||
fstr = str(fromfile) | ||
if self.ignore_file(fstr): | ||
continue | ||
assert fstr.startswith(frompath) | ||
tofile = pathlib.Path(fstr.replace(frompath, topath, 1)) | ||
if not tofile.exists(): | ||
self.issues.append(f'Missing file: {tofile}') | ||
elif check_contents: | ||
if not filecmp.cmp(fromfile, tofile, shallow=False): | ||
self.issues.append(f'File contents differ: {fromfile}') | ||
|
||
def print_results(self) -> None: | ||
if self.issues: | ||
print('Build differences detected') | ||
for i in self.issues: | ||
print(i) | ||
else: | ||
print('No differences detected.') | ||
|
||
def check_output(self) -> None: | ||
self.check_contents('buildrepro', 'buildrepro.1st', True) | ||
self.check_contents('buildrepro.1st', 'buildrepro', False) | ||
|
||
def run(options: T.Any) -> None: | ||
rt = ReproTester(options) | ||
try: | ||
sys.exit(rt.run()) | ||
except FileNotFoundError as e: | ||
print(e) | ||
sys.exit(1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo: it should be
--intermediaries