Use pip repos with different names between different python versions

[michaeljs1990]

🐞 bug report

Affected Rule

This seems to be an "issue" when you using py_library and add a dependency to it that is in a pip repo not using your default python version. Hopefully I explained that correctly but I also opened a PR to make it easy to reproduce #2529.

Is this a regression?

I don't think so I have another version of rules python in use 0.32.2 I believe and it has the same issue.

Description

#2529 described better here but when using multiple repos and python versions and trying to use the library that contains a requirement from the non default and building it things blow up and look like this.

ERROR: /private/var/tmp/_bazel_schuettm/e053d0eaf0a843e9f76bdb61766af4b7/external/rules_python++pip+pypi_3_10_only/flask/BUILD.bazel:5:12: configurable attribute "actual" in @@rules_python++pip+pypi_3_10_only//flask:pkg doesn't match this configuration: No matching wheel for current configuration's Python version.

The current build configuration's Python version doesn't match any of the Python
wheels available for this distribution. This distribution supports the following Python
configuration settings:
    //_config:is_python_3.11

To determine the current configuration's Python version, run:
    `bazel config <config id>` (shown further below)

and look for one of:
    @@rules_python+//python/config_settings:python_version
    @@rules_python+//python/config_settings:pip_whl
    @@rules_python+//python/config_settings:pip_whl_glibc_version
    @@rules_python+//python/config_settings:pip_whl_muslc_version
    @@rules_python+//python/config_settings:pip_whl_osx_arch
    @@rules_python+//python/config_settings:pip_whl_osx_version
    @@rules_python+//python/config_settings:py_freethreaded
    @@rules_python+//python/config_settings:py_linux_libc

If the value is missing, then the default value is being used, see documentation:
https://rules-python.readthedocs.io/en/latest/api/rules_python/python/config_settings

This instance of @@rules_python++pip+pypi_3_10_only//flask:pkg has configuration identifier e62ee83. To inspect its configuration, run: bazel config e62ee83.

For more help, see https://bazel.build/docs/configurable-attributes#faq-select-choose-condition.

Use --verbose_failures to see the command lines of failed build steps.
ERROR: Analysis of target '//tests:my_lib_3_10_only_test' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.197s, Critical Path: 0.02s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully
//tests:test_versions                                                 NO STATUS

Executed 0 out of 1 test: 1 was skipped.

🔬 Minimal Reproduction

#2529 and just run bezel test //... inside examples/multi_python_versions.

🔥 Exception or Error

See above

🌍 Your Environment

Operating System:

This is on a Mac but I believe it is independent of OS.

Output of bazel version:

I have hit this on 7.x and 8..x

Rules_python version:

0.32.0 -> main

[aignas]

Just what I wrote in #2529:

Just to write here what I wrote in a related thread on Slack. I think what is happening here is:

• The default python toolchain version is 3.11 and if you build any target in the repo that is not using transitions, //python/version:python_version will just use that.
• This means that the flask here will be evaluated and will fail because there is no set of dependencies for 3.11 in the deps list.

There are ways to work around it:

• Set the target_compatible_with for this target, see the suggestion below.
• Set tags = ["manual"] and create a separate target that is transitioning the my_310_only_lib to something a build configuration where python_version = 3.10.
• Add a pip.parse entry for the default python toolchain version.
• Make 3.10 the default toolchain version by correctly setting is_default = True in one of the python.toolchain invocations.

The TCW snippet:

    target_compatible_with = select({
        "@rules_python//python/config_settings:is_python_3.10": [],
        "//conditions:default": ["@platforms//:incompatible"],
    })

Not sure exactly how rules_python can help here best. The error allows to ensure that the packages are getting the versions that they expect, but I see how this might be adding unnecessary work on the consumer side.

I can only think of the following two ideas:

• Create a transitionable py_library implementation. This has a drawback that for each py_library target we would create an extra target and that would increase the total number of targets by a factor of 2-ish.
• Add a target_compatible_with automatically to the pypi hub repo targets. This means that bazel build //... would succeed without doing anything if the --//python/config_settings:python_version is not set to the same version as the python_version from the hub repo.

None of them is ideal, but I am leaning towards the second one. I am also thinking that if the #1708 is fixed, then this could be a lesser issue.