Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Do not merge] dogfood local environments for pantsbuild/pants #17043

Closed
wants to merge 1 commit into from

Conversation

Eric-Arellano
Copy link
Contributor

A common desire is to not use system Python interpreters on macOS because they tend to be broken. However, in CI, we do need to use those rather than Pyenv.

This PR demonstrates how to do that by defining 2 local_environment targets for macOS, and then overriding what the env name macos refers to in CI. It's necessary to override names like this to avoid Pants erroring that there is ambiguity with what __local__ refers to because >1 target sets compatible_platforms to the same values for macOS.

How to change the env values locally

If a user does not like what //:macos_local_env is set to, then they would need to override [environments].names, similar to pants.ci.toml.

If they cannot use //:macos_ci_env, then this implies they would need to define a new target in BUILD with their configuration, but not give it a name inside pants.toml so that it's ignored by default. That's a bummer in that it's harder to change than before, as they need to check in their change to BUILD. If this becomes a problem, we could explore:

  1. Giving an option in [environments] to let people override the value of a particular environment.
  2. Some type of "private" BUILD file that Pants uses, but is not checked in. E.g. BUILD.rc. We could ban this being anything more than environment targets.

Future direction: more predicates

We could avoid needing to override the names option if we had better predicates, like "use this environment only if PYENV is defined", because it would mean we don't have ambiguity anymore. Even with 2 macOS environments, only one would match each predicate.

Rejected idea: local_override option

We could let the user tell us what __local__ resolves to via an option. That would allow the user to define both macOS local_environment targets while avoiding ambiguity.

However, this does not work out well. It requires that the local_override always be set to avoid ambiguity. But there is no way to set that value safely in pants.toml because we don't know the platform of who will be running Pants. For example, this option would break any Linux users:

[environments]
local_override = "macos_local"

So, we could avoid that by using an option like platforms_to_local_override:

[environments.platforms_to_local_override]
macos_linux_x86_64= "macos_local"

That works well! But, it falls apart when we add future predicates like making by OS name (e.g. centos), because we would need to consider every single predicate in the option.

Only a demo

We do not actually merge this PR because the risk that a pantsbuild/pants developer doesn't have Pyenv is too high. For example, they could be using Homebrew, which would show up via <PATH>. We don't want to break them.

[ci skip-rust]

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]
Eric-Arellano added a commit that referenced this pull request Sep 28, 2022
There's no need for us to set up local environments, now that local environments are only to _override_ values from the subsystems. (Before, we were going to deprecate the subsystems.)

We may in the future want to add a macos_arm64 environment. That is safe for us to do without adding other local environments thanks to #16983.

See #17043 for an alternative way we could be leveraging environments locally.

[ci skip-rust]
Eric-Arellano added a commit that referenced this pull request Sep 29, 2022
See #17043 for a demo of this, and an explanation why we take this approach.

[ci skip-rust]
[ci skip-build-wheels]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant