From 0efec379b9b5d77d783b214395d62dff52d91a89 Mon Sep 17 00:00:00 2001 From: Anisse Astier Date: Tue, 21 May 2024 17:17:53 +0200 Subject: [PATCH] pex generation: remove use of lock file and wheels The way we generate pex files broke when we added more dependencies (we use numpy for graph generation). The choice was to build from source, without wheels; but for that we need to be able to re-build numpy, and this can be time-consuming. If we remove the --no-wheel option, we go back to the exact versions pinned in the generated requirements/base.txt; this follows the initial intent of having reproducible builds for the pex. But this does not work because the current pex implementation does not support locking versions like this, while also having a local package (here hwbench itself) in a directory (and not a wheel which has a hash). We get this error: ERROR: Can't verify hashes for these file:// requirements because they point to directories: See this pex issues and the other linked ones: https://github.com/pex-tool/pex/issues/429 ; currently the pex project is building its own lock file format to better fit its needs, but that would not help us here. So this change relaxes the constraints a bit: no more lock file. The pex builds won't be reproducible, and hopefully it won't be an issue since the artifacts will be archived. And no more wheels: this should make the pexes generation significantly faster. Also upgrade the dependency resolver to be the same one we use with pip-compile (backtracking), so that we'd get a similar result if pex and requirements/base.txt are generated at the same time. Change-Id: If6d677838f86880776a125c1f5676cbf1b08ddbb --- tox.ini | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tox.ini b/tox.ini index 6e96bba..219668f 100644 --- a/tox.ini +++ b/tox.ini @@ -20,7 +20,7 @@ commands = # Creates a source archive in sdist/. {envpython} setup.py sdist --dist-dir=sdist --format=gztar # Puts binary archives in dist/. - {envpython} setup.py bdist_pex --bdist-dir=dist --pex-args='--disable-cache --no-wheel -r requirements/base.txt' --bdist-all + {envpython} setup.py bdist_pex --bdist-dir=dist --pex-args='--disable-cache -r requirements/base.in --resolver-version pip-2020-resolver' --bdist-all # Linter environment [testenv:lint]