Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: prefer python_version attribute for specifying python version #2565

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions docs/_includes/py_console_script_binary.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ py_console_script_binary(
)
```

Or for more advanced setups you can also specify extra dependencies and the
#### Specifying extra dependencies
You can also specify extra dependencies and the
exact script name you want to call. It is useful for tools like `flake8`, `pylint`,
`pytest`, which have plugin discovery methods and discover dependencies from the
PyPI packages available in the `PYTHONPATH`.
Expand All @@ -34,17 +35,26 @@ py_console_script_binary(
)
```

A specific Python version can be forced by using the generated version-aware
wrappers, e.g. to force Python 3.9:
#### Using a specific Python version

A specific Python version can be forced by passing the desired Python version, e.g. to force Python 3.9:
```starlark
load("@python_versions//3.9:defs.bzl", "py_console_script_binary")
load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary")

py_console_script_binary(
name = "yamllint",
pkg = "@pip//yamllint",
python_version = "3.9"
)
```

#### Using a specific Python Version directly from a Toolchain
:::{deprecated} 1.1.0
The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules.
i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` and `load("@python_versions//3.11:defs.bzl", "py_test")`

You should instead specify the desired python version with `python_version`; see above example.
:::
Alternatively, the [`py_console_script_binary.binary_rule`] arg can be passed
the version-bound `py_binary` symbol, or any other `py_binary`-compatible rule
of your choosing:
Expand Down
64 changes: 56 additions & 8 deletions docs/toolchains.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,9 +116,9 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(python_version = "3.12")

# BUILD.bazel
load("@python_versions//3.12:defs.bzl", "py_binary")
load("@rules_python//python:py_binary.bzl", "py_binary")

py_binary(...)
py_binary(..., python_version="3.12")
```

### Pinning to a Python version
Expand All @@ -132,21 +132,59 @@ is most useful for two cases:
typically in a mono-repo situation.

To configure a submodule with the version-aware rules, request the particular
version you need, then use the `@python_versions` repo to use the rules that
force specific versions:
version you need when defining the toolchain:

```starlark
# MODULE.bazel
python = use_extension("@rules_python//python/extensions:python.bzl", "python")

python.toolchain(
python_version = "3.11",
)
use_repo(python, "python_versions")
use_repo(python)
```

Then use the `@rules_python` repo in your BUILD file to explicity pin the Python version when calling the rule:

```starlark
# BUILD.bazel
load("@rules_python//python:py_binary.bzl", "py_binary")

py_binary(..., python_version = "3.11")
py_test(..., python_version = "3.11")
```

Then use e.g. `load("@python_versions//3.11:defs.bzl", "py_binary")` to use
the rules that force that particular version. Multiple versions can be specified
and use within a single build.
Multiple versions can be specified and used within a single build.

```starlark
# MODULE.bazel
python = use_extension("@rules_python//python/extensions:python.bzl", "python")

python.toolchain(
python_version = "3.11",
is_default = True,
)

python.toolchain(
python_version = "3.12",
)

# BUILD.bazel
load("@rules_python//python:py_binary.bzl", "py_binary")
load("@rules_python//python:py_test.bzl", "py_test")

# Defaults to 3.11
py_binary(...)
py_test(...)

# Explicitly use Python 3.11
py_binary(..., python_version = "3.11")
py_test(..., python_version = "3.11")

# Explicitly use Python 3.12
py_binary(..., python_version = "3.12")
py_test(..., python_version = "3.12")
```

For more documentation, see the bzlmod examples under the {gh-path}`examples`
folder. Look for the examples that contain a `MODULE.bazel` file.
Expand All @@ -159,6 +197,16 @@ The `python.toolchain()` call makes its contents available under a repo named
Remember to call `use_repo()` to make repos visible to your module:
`use_repo(python, "python_3_11")`


:::{deprecated} 1.1.0
The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules.
i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")`

Usages of them should be changed to load the regular rules directly;
i.e. Use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the python version you defined in your toolchain. {ref}`Library modules with version constraints`
:::


#### Toolchain usage in other rules

Python toolchains can be utilized in other bazel rules, such as `genrule()`, by
Expand Down
120 changes: 79 additions & 41 deletions examples/multi_python_versions/tests/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,90 +1,98 @@
load("@bazel_skylib//rules:copy_file.bzl", "copy_file")
load("@bazel_skylib//rules:diff_test.bzl", "diff_test")
load("@bazel_skylib//rules:write_file.bzl", "write_file")
load("@python//3.10:defs.bzl", py_binary_3_10 = "py_binary", py_test_3_10 = "py_test")
load("@python//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_test")
load("@python//3.8:defs.bzl", py_binary_3_8 = "py_binary", py_test_3_8 = "py_test")
load("@python//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test")
load("@pythons_hub//:versions.bzl", "MINOR_MAPPING", "PYTHON_VERSIONS")
load("@rules_python//python:py_binary.bzl", "py_binary")
load("@rules_python//python:py_test.bzl", "py_test")
load("@rules_python//python:versions.bzl", DEFAULT_MINOR_MAPPING = "MINOR_MAPPING", DEFAULT_TOOL_VERSIONS = "TOOL_VERSIONS")
load("@rules_python//python/private:text_util.bzl", "render") # buildifier: disable=bzl-visibility
load("@rules_shell//shell:sh_test.bzl", "sh_test")

copy_file(
name = "copy_version",
src = "version.py",
out = "version_default.py",
is_executable = True,
)

# NOTE: We are testing that the `main` is an optional param as per official
# docs https://bazel.build/reference/be/python#py_binary.main
################################################################################
# Defining py_binary with different versions

py_binary(
name = "version_default",
srcs = ["version_default.py"],
srcs = ["version.py"],
main = "version.py",
# NOTE: # The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel
)

py_binary_3_8(
py_binary(
name = "version_3_8",
srcs = ["version.py"],
main = "version.py",
# NOTE: The value specified with python_version must be a toolchain configured to match the Python version
python_version = "3.8"
)

py_binary_3_9(
py_binary(
name = "version_3_9",
srcs = ["version.py"],
main = "version.py",
python_version = "3.9"
)

py_binary_3_10(
py_binary(
name = "version_3_10",
srcs = ["version.py"],
main = "version.py",
python_version = "3.10"
)

py_binary_3_11(
py_binary(
name = "version_3_11",
srcs = ["version.py"],
main = "version.py",
python_version = "3.11"
)

################################################################################
# Defining py_test with different versions with deps

py_test(
name = "my_lib_default_test",
srcs = ["my_lib_test.py"],
main = "my_lib_test.py",
deps = ["//libs/my_lib"],
# NOTE: # The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel
)

py_test_3_8(
py_test(
name = "my_lib_3_8_test",
srcs = ["my_lib_test.py"],
main = "my_lib_test.py",
deps = ["//libs/my_lib"],
python_version = "3.8"
)

py_test_3_9(
py_test(
name = "my_lib_3_9_test",
srcs = ["my_lib_test.py"],
main = "my_lib_test.py",
deps = ["//libs/my_lib"],
python_version = "3.9"
)

py_test_3_10(
py_test(
name = "my_lib_3_10_test",
srcs = ["my_lib_test.py"],
main = "my_lib_test.py",
deps = ["//libs/my_lib"],
python_version = "3.10"
)

py_test_3_11(
py_test(
name = "my_lib_3_11_test",
srcs = ["my_lib_test.py"],
main = "my_lib_test.py",
deps = ["//libs/my_lib"],
)

################################################################################
# Sanity check that py_test is using the expected python verion

copy_file(
name = "copy_version_test",
src = "version_test.py",
Expand All @@ -95,39 +103,47 @@ copy_file(
py_test(
name = "version_default_test",
srcs = ["version_default_test.py"],
env = {"VERSION_CHECK": "3.9"}, # The default defined in the WORKSPACE.
# The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel
env = {"VERSION_CHECK": "3.9"},
)

py_test_3_8(
name = "version_3_8_test",
py_test(
name = "version_3_9_test",
srcs = ["version_test.py"],
env = {"VERSION_CHECK": "3.8"},
env = {"VERSION_CHECK": "3.9"},
main = "version_test.py",
python_version = "3.9" # We can explicity mention the python version instead of using the default python version
)

py_test_3_9(
name = "version_3_9_test",
py_test(
name = "version_3_8_test",
srcs = ["version_test.py"],
env = {"VERSION_CHECK": "3.9"},
env = {"VERSION_CHECK": "3.8"},
main = "version_test.py",
python_version = "3.8"
)

py_test_3_10(
py_test(
name = "version_3_10_test",
srcs = ["version_test.py"],
env = {"VERSION_CHECK": "3.10"},
main = "version_test.py",
python_version = "3.10"
)

py_test_3_11(
py_test(
name = "version_3_11_test",
srcs = ["version_test.py"],
env = {"VERSION_CHECK": "3.11"},
main = "version_test.py",
python_version = "3.11"
)

################################################################################
# Cross Version Target Usage

py_test(
name = "version_default_takes_3_10_subprocess_test",
name = "python_version_default_takes_3_10_subprocess_test",
srcs = ["cross_version_test.py"],
data = [":version_3_10"],
env = {
Expand All @@ -138,8 +154,8 @@ py_test(
main = "cross_version_test.py",
)

py_test_3_10(
name = "version_3_10_takes_3_9_subprocess_test",
py_test(
name = "python_version_3_10_takes_3_9_subprocess_test",
srcs = ["cross_version_test.py"],
data = [":version_3_9"],
env = {
Expand All @@ -148,25 +164,47 @@ py_test_3_10(
"VERSION_CHECK": "3.10",
},
main = "cross_version_test.py",
python_version = "3.10"
)

################################################################################
# Using Multiple Python Versions with sh_test

sh_test(
name = "version_test_binary_default",
name = "version_test_binary_3_8",
srcs = ["version_test.sh"],
data = [":version_default"],
data = [":version_3_8"],
env = {
"VERSION_CHECK": "3.9", # The default defined in the WORKSPACE.
"VERSION_PY_BINARY": "$(rootpaths :version_default)",
# The python version is transiently obtained from the py_binary target :version_3_8; which in this case is 3.8
"VERSION_CHECK": "3.8",
"VERSION_PY_BINARY": "$(rootpaths :version_3_8)",
},
)

copy_file(
name = "copy_version",
src = "version.py",
out = "version_default.py",
is_executable = True,
)

# NOTE: We are testing that the `main` is an optional param as per official
# docs https://bazel.build/reference/be/python#py_binary.main
py_binary(
name = "version_default",
srcs = ["version_default.py"],
# The default python_version used by this rule is defined in the WORKSPACE / MODULE.bazel
)

sh_test(
name = "version_test_binary_3_8",
name = "version_test_binary_default",
srcs = ["version_test.sh"],
data = [":version_3_8"],
data = [":version_default"],
env = {
"VERSION_CHECK": "3.8",
"VERSION_PY_BINARY": "$(rootpaths :version_3_8)",
# The python version is transiently obtained from the py_binary target :version_default,
# which happens to be the default python_version defined by the toolchain in workspace/MODULE.bazel
"VERSION_CHECK": "3.9",
"VERSION_PY_BINARY": "$(rootpaths :version_default)",
},
)

Expand Down