Switch to the hatchling build backend.
#2331
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This should elminate flaky CI due to ephemeral dirs in the source root created by the setuptools build backend which were not multi-process safe. The `hatchling` build backend supports metadata plugins and it performs isolated builds making it a good replacement for the prior setuptools hack for effecting dynamic `Requires-Python` to enable Python 3.13 shadow-support. Unfortunately, there is no nice way to specify an in-tree plugin; so we actually use a very thin in-tree build backend that wraps `hatchling.build`, to allow discovery of our in-tree plugin.
jsirois
commented
Jan 17, 2024
| @@ -451,7 +452,7 @@ def spawn_download_distributions( | |||
|
|
|||
| if not build_isolation: | |||
| download_cmd.append("--no-build-isolation") | |||
| extra_env.update(PEP517_BACKEND_PATH=os.pathsep.join(sys.path)) | |||
| pex_extra_sys_path.extend(sys.path) | |||
There was a problem hiding this comment.
Now that Pex uses an in-tree build backend, it unearthed this latent bug where PEP517_BACKEND_PATH would get stomped with the backend-path. As such, both here in spawn download and lower in spawn install wheel, we now pass non-isolated-build sys.path via PEX_EXTRA_SYS_PATH instead.
jsirois
commented
Jan 17, 2024
| @@ -46,7 +47,12 @@ def build_pex_pex(output_file: PurePath, verbosity: int = 0) -> None: | |||
| subprocess.run(args, check=True) | |||
|
|
|||
|
|
|||
| def describe_git_rev() -> str: | |||
| def describe_rev() -> str: | |||
| if not os.path.isdir(".git") and os.path.isfile("PKG-INFO"): | |||
There was a problem hiding this comment.
This was always a bug, I just never tried unpacking our sdist and doing a build from there before. That now works.
jsirois
commented
Jan 17, 2024
| subprocess.check_call( | ||
| args=[build_system.venv_pex.pex, "-c", "import {}".format(build_system.build_backend)] | ||
| args=[build_system.venv_pex.pex, "-c", "import {}".format(build_system.build_backend)], | ||
| env=build_system.env, |
There was a problem hiding this comment.
This env carries the backend-path via PEX_EXTRA_SYS_PATH which is needed to have the build see our now in-tree build backend.
jsirois
commented
Jan 17, 2024
| @@ -55,5 +55,6 @@ def test_load_build_system_pyproject_custom_repos( | |||
| build_system = load_build_system(current_target, custom_resolver, pex_project_dir) | |||
| assert isinstance(build_system, BuildSystem) | |||
| subprocess.check_call( | |||
| args=[build_system.venv_pex.pex, "-c", "import {}".format(build_system.build_backend)] | |||
| args=[build_system.venv_pex.pex, "-c", "import {}".format(build_system.build_backend)], | |||
| env=build_system.env, | |||
There was a problem hiding this comment.
kaos
approved these changes
Jan 17, 2024
The default Pip for Python 2.7 (19.x) was too old to handle in-tree build backends.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This should elminate flaky CI due to ephemeral dirs in the source root
created by the setuptools build backend which were not multi-process
safe.
The
hatchlingbuild backend supports metadata plugins and it performsisolated builds making it a good replacement for the prior setuptools
hack for effecting dynamic
Requires-Pythonto enable Python 3.13shadow-support. Unfortunately, there is no nice way to specify an
in-tree plugin; so we actually use a very thin in-tree build backend
that wraps
hatchling.build, to allow discovery of our in-tree plugin.