Add support for consuming multiple Python lockfiles/resolves per repository. #11165
Comments
benjyw
changed the title
|
Generating multiple lockfiles is out of scope, this ticket assumes that the user has some way of generating lockfiles via Poetry or similar. We may want to also support Pants actually generating lockfiles in the future. |
Eric-Arellano
changed the title
I'm not sure that I agree there. It is already awkward that our existing lockfile support requires manual steps, but particularly with multiple resolves/lockfiles, correctly including the appropriate targets in the command that generates the lockfile would be nearly impossible. Essentially, you'd have to manually run a query to "select all targets relevant to a particular resolve" (to make this not-impossible, we'd probably need to add support to I think that generating the |
|
Closing in favor of #12314. |
Part of #11165 and builds off of #12703. Rather than having a single option `[python-setup].experimental_lockfile`, users set `[python-setup].experimental_resolves_to_lockfiles` to define 0-n "named resolves" that associate a lockfile with a name: ```toml [python-setup] experimental_resolves_to_lockfiles = { lock1 = "lock1.txt", lock2 = "lock2.txt" } ``` Then, individual `pex_binary` targets can specify which resolve to use: ```python pex_binary(name="reversion", entry_point="reversion.py", experimental_resolve="lock1") ``` In a followup, we'll add a mechanism to set the default resolve. Users can generate that lockfile with `./pants generate-lockfiles` (all resolves) or `./pants generate-lockfiles --resolve=<name>`: ``` ❯ ./pants generate-lockfiles --resolve=lock1 --resolve=lock2 15:55:56.60 [INFO] Completed: Generate lockfile for lock1 15:55:56.61 [INFO] Completed: Generate lockfile for lock2 15:55:57.02 [INFO] Wrote lockfile for the resolve `lock1` to lock1.txt 15:55:57.02 [INFO] Wrote lockfile for the resolve `lock2` to lock2.txt ``` Then, it will be consumed with `./pants package` and `./pants run`. Pants will extract the proper subset from that lockfile, meaning that the lockfile can safely be a superset of what is used for the particular build. ``` ❯ ./pants package build-support/bin: ... 15:56:33.87 [INFO] Completed: Installing lock1.txt for the resolve `lock1` 15:56:34.39 [INFO] Completed: Installing lock2.txt for the resolve `lock2` 15:56:34.48 [INFO] Completed: Extracting 1 requirement to build build-support.bin/generate_user_list.pex from lock1_lockfile.pex: pystache==0.5.4 ... ``` If the lockfile is incompatible, we will (soon) warn or error with instructions to either use a new resolve or regenerate the lockfile. In followups, this field will be hooked up to other targets like `python_awslambda` and `python_tests`. We will likely also add a new field `compatible_resolves` to `python_library`, per #12714, which is a list of resolves. "Root targets" like `python_tests` and `pex_binary` will validate that all their dependencies are compatible. When you operate directly on a `python_library` target, like running MyPy on it, we will choose any of the possible resolves. You will be able to set your own default for this field. [ci skip-rust] [ci skip-build-wheels]
https://docs.google.com/document/d/1sr1O7W9517w8YbdzfE56xvDb72LafK94Fwxm0juIRFc/edit#
The text was updated successfully, but these errors were encountered: