Allow root module to customize toolchains

[rickeylev]

We've had several requests where users want to modify something about the toolchain definitions that are generated for the hermetic runtimes. Typically these are small tweaks. Allowing other modules to tweak toolchains is a no-go, but allowing the root module to change things is reasonable.

The "simple" option for customizing toolchains is to add args to python_register_toolchains and python.toolchain.

The two main issues I see with adding args is:

1. bzlmod APIs tend to be "written in stone". They happen very early in the build process, so we have limited options for evolving the API. In this case, this is somewhat tempered by only the root module being allowed to use it, but it still makes me uneasy.
2. Toolchains tend to have unstable APIs. Most parts of them are just implementation details for the rules. For example, adding a single arg to customize the bootstrap template made sense last year, but now there are multiple template files. Similarly, the stub_shebang attribute is slated for removal with the introduce of --bootstrap_impl=script.

Adding args to python.toolchain is, I think, a bad idea. Having something else, e.g. python.root_config might make a bit more sense. That said, args on python.toolchain() make it easy to associate something to a particular toolchain, while python.root_config would need some arg to specify which toolchain it wanted to restrict the setting to.

This issue is to collect the use cases and try and figure out some options.

Things people want to change:

• Overriding bootstrap template

  • Example: feat: Allow custom bootstrap template in python_repository #2032
    • Goal: They want to implement a separate bootstrap
    • The PR proposes adding args to e.g. python_register_toolchains().
    • Issues: (1) Would need to be exposed via bzmod apis. (2) There are 4 bootstrap files: python_bootstrap_template (legacy system python bootstrap), stage1 script bootstrap, stage2 bootstrap, and zip main bootstrap. Plus the launcher exe thing for windows.
    • Proposal: Indirect the references to the templates using flags.

• Customizing stub_shebang: https://bazelbuild.slack.com/archives/CA306CEV6/p1721464127008299

  • Goal: They want to set -S PYTHONNOUSERSITE=1 for all their Python invocations
  • Proposal 1: Have a flag override for stub_shebang
  • Proposal 2: Add interpreter_args and make env populate values into the bootstrap.
  • Proposal 3: Have flags for "default interpreter args" and "default interpreter env"

• The register_coverage_tool arg

  • Goal: Having coverage included as part of the toolchain is unnecessary if a user test provides the coverage library itself

• Control whether local or hermetic runtimes are used

  • Goal: Provide a way for the root module to decide whether a local or hermetic toolchain is used.
  • Basically a local=True|False arg when defining a toolchain. It decides whether the hermetic runtime repo rule or local runtime repo rule is used under the hood.
  • Proposal: Have a flag that acts as a constraint for matching toolchains.

• Control whether runtimes are in-build or platform runtimes

  • Goal: A platform runtime is cheaper to setup; while not compatible with RBE or sandboxing, that's fine if you aren't using RBE or sandboxing.
  • This mostly applies to local runtimes, but could also apply to hermetic.
  • Basically, have an inbuild=True|False arg when defining a toolchain. It controls whether the underlying py_runtime() targets generated use interpreter= or interpreter_path=

• Control constraints of toolchains

  • Goal: Allow registering both local and hermetic runtimes. Using constraints, local or platform toolchains can be used for local invocations, and hermetic for RBE.

• Control what toolchain versions are available

  • Goal: An infrastructure team wants to ensure only e.g. Python 3.11 (or a small set of proscribed versions) is usable at their company.

[aignas]

Wanted to create a separate issue for this but realized that this is already there. We could just create a python.override tag class for the majority of the things listed above, but for starters we could start with customising the URLs for the python toolchain override.

Currently the TOOL_VERSIONS used by the extension cannot be overridden via MODULE.bazel. The only way users have right now is to use the bazel_downloader config, but that may give hard to debug errors and still does not allow users to add Python builds for extra platforms that are not present on the indygreg website.

Initial solution could be a python.override tag class that can override the URLs/structs for python toolchains that are used. The design of the API should allow for future extension for different overrides:

• restricting allowed python versions
• setting the x.y->x.y.z mapping
• disable registering all versions
• etc