Add [python].enable_resolves and `[python].experimental_default_res…

…olve` (#13967)

Closes #12742.

Similar to #13925, this adds a default of `{"python-default": "3rdparty/python/default_lock.txt"}`. It also lets the user set the default resolve via `--experimental-default-resolve`.

We don't add `--default-compatible-resolves`, as decided in #13902.

## Default file name

The resolve must be unambiguous with JVM, hence using `python-default` instead of `default`.

Everyone agreed in Slack that we should use `python` instead of `py`, e.g. `3rdparty/python` over `3rdparty/py`. The clarity is worth it.

It's unclear to me if we want the file to be called `default_lock.txt`. In JVM, it's called `default.lock`. We could do that here too? I only went with `.txt` to make it a bit more clear to users that they can use these lockfiles just like a normal `requirements.txt` - Python has a strong convention for this.

## Deprecation plan: `--enable-resolves`

We want to require using resolves in the future, as we believe it is important for users to have secure and reproducible builds. But we can't switch it on immediately.

At the same time, to make the feature useful, it's really useful for us to be able to set defaults.

To bridge the gap, `--enable-resolves` feature gates everything. When set, we can act how we will in the future, e.g. assuming that a default resolve must always be set. When unset, we use the status quo. In Pants 2.10, we will deprecate not setting this option so that we can make the default in Pants 2.11 be to use resolves.

[ci skip-rust]
[ci skip-build-wheels]

src/python/pants/backend/python/goals/lockfile.py

@@ -269,6 +269,9 @@ class _UserLockfileRequests(Collection[PythonLockfileRequest]):
 async def setup_user_lockfile_requests(
     requested: _SpecifiedUserResolves, all_targets: AllTargets, python_setup: PythonSetup
 ) -> _UserLockfileRequests:
+    if not python_setup.enable_resolves:
+        return _UserLockfileRequests()
+
     # First, associate all resolves with their consumers.
     resolves_to_roots = defaultdict(list)
     for tgt in all_targets:

src/python/pants/backend/python/subsystems/setup.py

@@ -87,7 +87,8 @@ def register_options(cls, register):
                 "See https://pip.pypa.io/en/stable/user_guide/#constraints-files for more "
                 "information on the format of constraint files and how constraints are applied in "
                 "Pex and pip.\n\n"
-                "Mutually exclusive with `[python].experimental_lockfile`."
+                "Mutually exclusive with `[python].experimental_lockfile` and "
+                "`[python].enable_resolves`."
             ),
         )
         register(
@@ -113,29 +114,56 @@ def register_options(cls, register):
             # TODO(#11719): Switch this to a file_option once env vars can unset a value.
             type=str,
             metavar="<file>",
-            mutually_exclusive_group="constraints",
+            mutually_exclusive_group="lockfile",
             help=(
                 "The lockfile to use when resolving requirements for your own code (vs. tools you "
                 "run).\n\n"
-                "This is highly experimental and will change, including adding support for "
-                "multiple lockfiles. This option's behavior may change without the normal "
-                "deprecation cycle.\n\n"
+                "This is highly experimental and will be replaced by `[python].enable_resolves`.\n\n"
                 "To generate a lockfile, activate the backend `pants.backend.experimental.python`"
                 "and run `./pants generate-user-lockfile ::`.\n\n"
-                "Mutually exclusive with `[python].requirement_constraints`."
+                "Mutually exclusive with `[python].requirement_constraints` and "
+                "`[python].enable_resolves`."
+            ),
+        )
+        register(
+            "--enable-resolves",
+            advanced=True,
+            type=bool,
+            default=False,
+            mutually_exclusive_group="lockfile",
+            help=(
+                "Set to true to enable the multiple resolves mechanism. See "
+                "`[python].experimental_resolves` for an explanation of this feature.\n\n"
+                "Mutually exclusive with `[python].experimental_lockfile` and "
+                "`[python].requirement_constraints`."
             ),
         )
         register(
             "--experimental-resolves",
             advanced=True,
             type=dict,
+            default={"python-default": "3rdparty/python/default_lock.txt"},
             help=(
-                "A mapping of logical names to lockfile paths used in your project, e.g. "
-                "`{ default = '3rdparty/default_lockfile.txt', py2 = '3rdparty/py2.txt' }`.\n\n"
+                "A mapping of logical names to lockfile paths used in your project.\n\n"
+                # TODO(#12314): explain how this feature works.
                 "To generate a lockfile, run `./pants generate-lockfiles --resolve=<name>` or "
                 "`./pants generate-lockfiles` to generate for all resolves (including tool "
                 "lockfiles).\n\n"
-                "This is highly experimental and will likely change."
+                "Only applies if `[python].enable_resolves` is true.\n\n"
+                "This option is experimental and may change without the normal deprecation policy."
+            ),
+        )
+        register(
+            "--experimental-default-resolve",
+            advanced=True,
+            type=str,
+            default="python-default",
+            help=(
+                "The default value used for the `experimental_resolve` and "
+                "`experimental_compatible_resolves` fields.\n\n"
+                "Only applies if `[python].enable_resolves` is true.\n\n"
+                "The name must be defined as a resolve in `[python].experimental_resolves`.\n\n"
+                "This option is experimental and may change without the normal deprecation policy."
             ),
         )
         register(
@@ -271,10 +299,18 @@ def requirement_constraints(self) -> str | None:
     def lockfile(self) -> str | None:
         return cast("str | None", self.options.experimental_lockfile)
 
+    @property
+    def enable_resolves(self) -> bool:
+        return cast(bool, self.options.enable_resolves)
+
     @property
     def resolves(self) -> dict[str, str]:
         return cast("dict[str, str]", self.options.experimental_resolves)
 
+    @property
+    def default_resolve(self) -> str:
+        return cast(str, self.options.experimental_default_resolve)
+
     @property
     def invalid_lockfile_behavior(self) -> InvalidLockfileBehavior:
         return cast(InvalidLockfileBehavior, self.options.invalid_lockfile_behavior)

src/python/pants/backend/python/target_types.py

@@ -125,20 +125,23 @@ def __init__(
 
 class PythonResolveField(StringField, AsyncFieldMixin):
     alias = "experimental_resolve"
-    # TODO(#12314): Figure out how to model the default and disabling lockfile, e.g. if we
-    #  hardcode to `default` or let the user set it.
     help = (
-        "The resolve from `[python].experimental_resolves` to use, if any.\n\n"
-        "This field is highly experimental and may change without the normal deprecation policy."
+        "The resolve from `[python].experimental_resolves` to use.\n\n"
+        "If not defined, will default to `[python].default_resolve`.\n\n"
+        "Only applies if `[python].enable_resolves` is true.\n\n"
+        "This field is experimental and may change without the normal deprecation policy."
+        # TODO: Document expectations for dependencies once we validate that.
     )
 
+    def value_or_default(self, python_setup: PythonSetup) -> str:
+        return self.value or python_setup.default_resolve
+
     def validate(self, python_setup: PythonSetup) -> None:
         """Check that the resolve name is recognized."""
-        if not self.value:
-            return None
-        if self.value not in python_setup.resolves:
+        resolve = self.value_or_default(python_setup)
+        if resolve not in python_setup.resolves:
             raise UnrecognizedResolveNamesError(
-                [self.value],
+                [resolve],
                 python_setup.resolves.keys(),
                 description_of_origin=f"the field `{self.alias}` in the target {self.address}",
             )
@@ -148,8 +151,11 @@ def resolve_and_lockfile(self, python_setup: PythonSetup) -> tuple[str, str] | N
 
         Error if the resolve name is invalid.
         """
+        if not python_setup.enable_resolves:
+            return None
         self.validate(python_setup)
-        return (self.value, python_setup.resolves[self.value]) if self.value is not None else None
+        resolve = self.value_or_default(python_setup)
+        return (resolve, python_setup.resolves[resolve])
 
 
 # -----------------------------------------------------------------------------------------------

src/python/pants/jvm/target_types.py

@@ -28,9 +28,8 @@ class JvmCompatibleResolvesField(StringSequenceField):
     alias = "compatible_resolves"
     required = False
     help = (
-        "The set of resolve names that this target is compatible with.\n\n"
+        "The set of resolves from `[jvm].resolves` that this target is compatible with.\n\n"
         "If not defined, will default to `[jvm].default_resolve`.\n\n"
-        "Each name must be defined as a resolve in `[jvm].resolves`.\n\n"
         # TODO: Document expectations for dependencies once we validate that.
     )
 
@@ -39,9 +38,8 @@ class JvmResolveField(StringField):
     alias = "resolve"
     required = False
     help = (
-        "The name of the resolve to use when building this target.\n\n"
+        "The resolve from `[jvm].resolves` to use when compiling this target.\n\n"
         "If not defined, will default to `[jvm].default_resolve`.\n\n"
-        "The name must be defined as a resolve in `[jvm].resolves`."
         # TODO: Document expectations for dependencies once we validate that.
     )