Skip to content

Update supported Python and package versions. #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tbreckle wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Update supported Python and package versions. #14

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .bazelignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
test
test_single_version
2 changes: 2 additions & 0 deletions .bazelrc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ build:python_3_9 --python=3.9
build:python_3_10 --python=3.10
build:python_3_11 --python=3.11
build:python_3_12 --python=3.12
build:python_3_13 --python=3.13
build:python_3_14 --python=3.14
build --config=python_3_12

common --flag_alias=use_bazel_tools_python_toolchains=//bazel/toolchains/python:bazel_tools_python_toolchains
Expand Down
12 changes: 9 additions & 3 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Eclipse Safe Open Vehicle Core (SCORE)

The [Eclipse Safe Open Vehicle Core](https://projects.eclipse.org/projects/automotive.score) project aims to develop an open-source core stack for Software Defined Vehicles (SDVs), specifically targeting embedded high-performance Electronic Control Units (ECUs).
Please check the [documentation](https://eclipse-score.github.io) for more information.
The source code is hosted at [GitHub](https://github.com/eclipse-score).
Expand All @@ -19,19 +20,23 @@ Want to contribute? You're welcoe and we're happy to accept your pull requests!
### Getting involved

#### Setup Phase

This phase is part of the eclipse Incubation Phase and shall establish all the processes needed for a safe development of functions. Only after this phase it will be possible to contribute code to the project. As the development in this project is driven by requirements, the processes and needed infrastructure incl. tooling will be established based on non-functional Stakeholder_Requirements<!-- TODO: fill link to correct page with requirements -->. During setup phase the contributions are Bug Fixes and Improvements (both on processes and infrastructure).

#### Bug Fixes and Improvements

Improvements are adding/changing processes and infrastructure, bug fixes can be also on development work products like code.
In case you want to fix a bug or contribute an improvement, please perform the following steps:
1) Create a PR by using the corresponding template ([Bugfix PR template](.github/PULL_REQUEST_TEMPLATE/bug_fix.md) or [Improvement PR template](.github/PULL_REQUEST_TEMPLATE/improvement.md)). Please mark your PR as draft until it's ready for review by the Committers (see the [Eclipse Foundation Project Handbook](https://www.eclipse.org/projects/handbook/#contributing-committers) for more information on the role definitions).

1) Create a PR by using the corresponding template ([Bugfix PR template](.github/PULL_REQUEST_TEMPLATE/bug_fix.md) or [Improvement PR template](.github/PULL_REQUEST_TEMPLATE/improvement.md)). Please mark your PR as draft until it's ready for review by the Committers (see the [Eclipse Foundation Project Handbook](https://www.eclipse.org/projects/handbook/#contributing-committers) for more information on the role definitions).
2) Initiate content review by opening a corresponding issue for the PR when it is ready for review. Review of the PR and final merge into the project repository is in responsibility of the Committers. Use the [Bugfix Issue template](.github/ISSUE_TEMPLATE/bug_fix.md) or [Improvement Issue template](.github/ISSUE_TEMPLATE/improvement.md) for this.

Please check here for our Git Commit Rules in the [Configuration_Tool_Guidelines](https://eclipse-score.github.io/score/process_description/guidelines/index.html).

Please use the [Stakeholder and Tool Requirements Template](https://eclipse-score.github.io/score/process_description/templates/index.html) when defining these requirements.

#### Additional Information

Please note, that all Git commit messages must adhere the rules described in the [Eclipse Foundation Project Handbook](https://www.eclipse.org/projects/handbook/#resources-commit).

Please find process descriptions here: [process description](https://eclipse-score.github.io/score/process_description/).
Expand All @@ -42,7 +47,8 @@ Please find process descriptions here: [process description](https://eclipse-sco

This repository uses Bazel `rules_python` pip integration plus a [custom pip hub implementation](bazel/rules/rules_python_pip_hub.bzl). Therefore, to add, remove or modify python pip dependencies, one should do as follows:

1. Update the dependency and its version under [requirements.in](third_party/pip/requirements.in);
1. Update the dependency and its version under [requirements.in_stable](third_party/pip/requirements.in_stable) and/or [requirements.in_legacy](third_party/pip/requirements.in_legacy) depending on which Python version the dependency is targeted to. See
[Tool Versions](README.md#tool-versions) for more information about legacy and stable python versions;
2. Lock pip requirements by executing `bazel run //third_party/pip:update.sh`. This will update all `requirements_lock_3_*.txt` files under `third_party/pip`;
3. Test the updated requirements by executing `bazel run //third_party/pip:test.sh`.

Expand All @@ -65,4 +71,4 @@ Other options are also available through other pip hub aliases, for example, the
#### Local quality check

Ideally, one should verify code quality locally before pushing changes to the CI. This avoids unecessary CI jobs and can even speed up development.
To do that, simply run [`scripts/run_all_tests.sh`](scripts/run_all_tests.sh).
To do that, simply run [`scripts/run_all_tests.sh`](scripts/run_all_tests.sh) from the repo root.
14 changes: 10 additions & 4 deletions MODULE.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# *******************************************************************************
module(
name = "score_bazel_tools_python",
version = "0.1.3",
version = "0.1.4",
compatibility_level = 0,
repo_name = "bazel_tools_python",
)
Expand All @@ -23,14 +23,16 @@ bazel_dep(name = "aspect_rules_lint", version = "1.4.4", dev_dependency = True)
bazel_dep(name = "buildifier_prebuilt", version = "8.2.0.2", dev_dependency = True)

# docs-as-code
bazel_dep(name = "score_tooling", version = "1.0.4", dev_dependency = True)
bazel_dep(name = "score_tooling", version = "1.0.5", dev_dependency = True)
bazel_dep(name = "platforms", version = "0.0.11", dev_dependency = True)

# Unfortunately bazel_skylib can not be dev_dependency because we use some of its libraries.
bazel_dep(name = "bazel_skylib", version = "1.7.1")
bazel_dep(name = "bazel_skylib", version = "1.9.0")

# Unfortunately rules_python can not be dev_dependency because we provide our pip hub using it.
bazel_dep(name = "rules_python", version = "1.4.1")
# Updating it to newer than 1.6.3 currently breaks our build due to changes in rules_python
# and we have to update score_score_tooling as well to accommodate those changes.
bazel_dep(name = "rules_python", version = "1.6.3")

# We patch rules_python patch to python coverage files.
# This enable us to select our own .coveragerc file when issuing a bazel coverage command.
Expand All @@ -48,6 +50,8 @@ PYTHON_VERSIONS = [
"3.10",
"3.11",
"3.12",
"3.13",
"3.14",
]

# By default, one can't add `dev_dependency = True` to `python` extension from
Expand Down Expand Up @@ -85,6 +89,8 @@ use_repo(
bazel_tools_python_python_3_10 = "python_3_10",
bazel_tools_python_python_3_11 = "python_3_11",
bazel_tools_python_python_3_12 = "python_3_12",
bazel_tools_python_python_3_13 = "python_3_13",
bazel_tools_python_python_3_14 = "python_3_14",
)

[
Expand Down
18 changes: 15 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,18 +26,30 @@ Each offered tool has it own README document and can be independently activated
| [pycoverage](quality/private/python/README.md) | Code coverage tool | aspect | Python | yes | yes |
| [pip_audit](quality/private/python/README.md) | Vulnerability checker | rule | Python | yes | yes |

## Tool Versions

Tool versions are handled differently depending on the Python version.
Legacy (end of life) Python versions use different tool versions than the stable supported Python versions.

Declaration of which Python versions are considered legacy can be found in the file
[bazel/toolchains/python/versions.bzl](bazel/toolchains/python/versions.bzl).

The tool versions are declared in [third_party/pip/requirements.in_legacy](third_party/pip/requirements.in_legacy) and
[third_party/pip/requirements.in_stable](third_party/pip/requirements.in_stable) respectively.

## How to use bazel_tools_python

The next sections explain the steps to achieve a proper config for each Bazel dependency manager. For a fully working setup, take a look at the [test workspace](test) as an example.

### Requirements

It's important to note that this repository does not supply python toolchains but only its pip dependencies. Therefore one must set up its own python toolchain. This repository support major python versions from `3.8` to `3.12`.
It's important to note that this repository does not supply python toolchains but only its pip dependencies.
Therefore one must set up its own python toolchain. This repository support major python versions from `3.8` to `3.14`.

Additionaly, one must have the following bazel repositories already in place:

- bazel_skylib >= 1.7.1
- rules_python >= 1.4.1
- bazel_skylib >= 1.9.0
- rules_python >= 1.7.0
Copy link
Contributor

@jannowotsch jannowotsch Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- rules_python >= 1.7.0
- rules_python >= 1.6.3

according to MODULES.bazel


### Select python pip hub version

Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ This project includes:

Known vulnerabilities may vary according to the selected Python version.

You can check our latest constraints against vulnerable package versions in our [requirements.in file](third_party/pip/requirements.in).
You can check our latest constraints against vulnerable package versions in our [requirements.in_stable file](third_party/pip/requirements.in_stable) and/or [requirements.in_legacy file](third_party/pip/requirements.in_legacy) depending on which Python version the dependency is targeted to. See [Tool Versions](README.md#tool-versions) for more information about legacy and stable python versions;

The following table lists all known vulnerabilities that could not be fixed:

Expand Down
47 changes: 41 additions & 6 deletions bazel/rules/rules_python_pip_hub.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,17 @@ _INDIRECTIONS_TYPES = [
"data",
]

def _get_direct_deps(repo_ctx):
"""Read a pip requirements.in file and return a list of direct dependencies."""
def _read_and_parse_requirements(repo_ctx, postfix):
"""Read a pip requirements file and return a list of dependencies (without version)."""
requirements = []

lines = repo_ctx.read(repo_ctx.attr.requirements_in).split("\n")
# Check if file is existing. If not just return an empty list.
requirements_path_str = str(repo_ctx.path(repo_ctx.attr.requirements_in)) + postfix
requirements_path = repo_ctx.path(requirements_path_str)
if not requirements_path.exists:
return []

lines = repo_ctx.read(requirements_path_str).split("\n")
for line in lines:
line = line.strip()
if not line:
Expand All @@ -48,12 +54,38 @@ def _get_direct_deps(repo_ctx):
if line.startswith("--"):
continue # Skip possible pip custom configurations.
if "==" not in line:
fail("Line '{}' in '{}' is missing a precise pinning via '=='.".format(line, str(repo_ctx.attr.requirements_in)) +
fail("Line '{}' in '{}' is missing a precise pinning via '=='.".format(line, str(repo_ctx.path(repo_ctx.attr.requirements_in)) + postfix) +
" While this is technically possible it violates our project best practices.")
requirements.append(line.split("==", 1)[0])

return requirements

def _get_direct_deps(repo_ctx):
"""Read pip requirements files and return a list of direct dependencies.

This will try to read the requirements.in file as well as
requirements.in_stable and requirements.in_legacy. In case one of those files
do not exist it will be simply skipped.
Copy link
Contributor

@jannowotsch jannowotsch Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
do not exist it will be simply skipped.
does not exist it will be simply skipped.


Returns:
list[str]: A list of unique direct dependencies found in the requirements files.
"""
requirements = {}

for requirement in _read_and_parse_requirements(repo_ctx, ""):
requirements[requirement] = True

for requirement in _read_and_parse_requirements(repo_ctx, "_stable"):
requirements[requirement] = True

for requirement in _read_and_parse_requirements(repo_ctx, "_legacy"):
requirements[requirement] = True

if not requirements:
fail("No direct dependencies found in any requirements file. The created pip hub will be empty.")

return list(requirements.keys())
Copy link
Contributor

@jannowotsch jannowotsch Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we anyhow return a list, would be possible to skip the dictionary step and just add the lists returned by _read_and_parse_requirements?


def _generate_single_indirection(dep, deps_to_config_map, indirection_type):
"""Generate the requested dependency indirection for a given dependency to config map."""
dep_name = pip_utils.normalize_name(dep)
Expand Down Expand Up @@ -141,10 +173,13 @@ This is needed so this custom pip hub can select which rules python pip hub shou

```
deps_to_config_map = {
"@rules_python_pip_hub_3_10": "@your_repo_name//label/to/your:config_setting_3_10",
"@rules_python_pip_hub_3_11": "@your_repo_name//label/to/your:config_setting_3_11",
"@rules_python_pip_hub_3_8": "@your_repo_name//label/to/your:config_setting_3_8",
"@rules_python_pip_hub_3_9": "@your_repo_name//label/to/your:config_setting_3_9",
"@rules_python_pip_hub_3_10": "@your_repo_name//label/to/your:config_setting_3_10",
"@rules_python_pip_hub_3_11": "@your_repo_name//label/to/your:config_setting_3_11",
"@rules_python_pip_hub_3_12": "@your_repo_name//label/to/your:config_setting_3_12",
"@rules_python_pip_hub_3_13": "@your_repo_name//label/to/your:config_setting_3_13",
"@rules_python_pip_hub_3_14": "@your_repo_name//label/to/your:config_setting_3_14",
}
```

Expand Down
10 changes: 9 additions & 1 deletion bazel/toolchains/python/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
load("@bazel_skylib//rules:common_settings.bzl", "bool_flag", "bool_setting")
load("@bazel_tools_python//bazel/toolchains/python:versions.bzl", "PYTHON_VERSIONS", "unsupported_python_configuration_error")
load("@bazel_tools_python//bazel/toolchains/python:versions.bzl", "LEGACY_PYTHON_VERSIONS", "PYTHON_VERSIONS", "legacy_python_warning", "unsupported_python_configuration_error")
load("@rules_python//python:defs.bzl", "py_runtime_pair")

bool_flag(
Expand Down Expand Up @@ -72,6 +72,14 @@ config_setting(
for version in PYTHON_VERSIONS
]

[
Copy link
Contributor

@jannowotsch jannowotsch Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to issue the warning only if the selected python version is a legacy one?

legacy_python_warning(
name = "legacy_python_{}_warning".format(version.replace(".", "_")),
python_version = version,
)
for version in LEGACY_PYTHON_VERSIONS
]

#############################################
# Incorrect toolchain configurations traps. #
#############################################
Expand Down
29 changes: 28 additions & 1 deletion bazel/toolchains/python/versions.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,21 @@

"""This module defines which python version are supported."""

PYTHON_VERSIONS = [
LEGACY_PYTHON_VERSIONS = [
"3.8",
"3.9",
]

STABLE_PYTHON_VERSIONS = [
"3.10",
"3.11",
"3.12",
"3.13",
"3.14",
]

PYTHON_VERSIONS = LEGACY_PYTHON_VERSIONS + STABLE_PYTHON_VERSIONS

def unsupported_python_configuration_error(custom_message):
default_message = """
----------------------------------------------------------------------
Expand All @@ -36,3 +43,23 @@ Please select a valid python version using rules_python's string_flag:
{custom_message}{default_message}
""".format(custom_message = custom_message, default_message = default_message)
return default_message

def _legacy_python_warning_impl(ctx):
"""Rule that prints a warning for legacy Python versions."""
python_version = ctx.attr.python_version

# buildifier: disable=print
print("⚠️ Python {} is a legacy version and may not be supported in future releases.".format(python_version))

return [DefaultInfo()]

legacy_python_warning = rule(
implementation = _legacy_python_warning_impl,
attrs = {
"python_version": attr.string(
mandatory = True,
doc = "The Python version string",
),
},
doc = "Rule that emits a warning for legacy Python versions during analysis.",
)
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ max-line-length=120
disallow_untyped_defs = false
explicit_package_bases = true
incremental = false
python_version = "3.9"
python_version = "3.10"

[tool.pylint.messages_control]
disable = [
Expand Down
1 change: 0 additions & 1 deletion quality/private/python/pycoverage/test/test_pycoverage_report_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,4 +60,3 @@ def test_pycoverage_reporter_main(mocker, caplog, reports_file, st_size, output_

if reports_file == "lcov_files_no_py_targets.tmp":
assert "No python coverage reports found." in caplog.text
return
13 changes: 1 addition & 12 deletions quality/private/python/tools/black_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,6 @@
WOULD_REFORMAT_MSG = "would reformat"


def _removeprefix(text: str, prefix: str) -> str:
"""Remove a certain prefix from a a given text.

This function is supposed to add backwards compartibility with python 3.8 as
python versions equal or greater than 3.9 already offer this as a built in.
"""
if text.startswith(prefix):
return text[len(prefix) :].strip()
return text


def get_black_command(aspect_arguments: python_tool_common.AspectArguments) -> t.List[str]:
"""Returns the command to run a black subprocess."""

Expand All @@ -54,7 +43,7 @@ def black_output_parser(tool_output: python_tool_common.SubprocessInfo) -> pytho

for line in tool_output.stderr.splitlines():
if line.startswith(WOULD_REFORMAT_MSG):
file = _removeprefix(line, WOULD_REFORMAT_MSG)
file = line.removeprefix(WOULD_REFORMAT_MSG)
findings += [
python_tool_common.Finding(
path=pathlib.Path(file),
Expand Down
14 changes: 1 addition & 13 deletions quality/private/python/tools/python_tool_common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -157,18 +157,6 @@ def execute_subprocess(
)


def _is_relative_to(path: pathlib.Path, root: pathlib.Path):
"""Helper function that mimics what pathlib.Path.is_relative_to does.

This is needed to ensure support for python 3.8.
"""
try:
path.relative_to(root)
return True
except ValueError:
return False


@dataclasses.dataclass
class AspectArguments: # pylint: disable=too-many-instance-attributes
"""Class that provides a clean and verified interface between aspect and runner."""
Expand All @@ -188,7 +176,7 @@ def resolve_paths(paths: t.List[str], prepend_path: str = "") -> t.Set[pathlib.P
resolved_paths = set()
for path in paths:
try:
if _is_relative_to(pathlib.Path(path), pathlib.Path(self.tool_root)):
if pathlib.Path(path).is_relative_to(pathlib.Path(self.tool_root)):
# This is the usual branch for local files or libraries.
# The code go through here when path is relative to the sandbox root.
resolved_paths.add(pathlib.Path(path).relative_to(self.tool_root).resolve(strict=True))
Expand Down
18 changes: 1 addition & 17 deletions quality/private/python/tools/test/test_black_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,22 +44,6 @@ def tearDown(self) -> None:
self.tmp_text_file_path.unlink()
self.tmp_json_file_path.unlink()

def test_black_removeprefix(self) -> None:
"""Test black _removeprefix for both text string cases."""
would_reformat_message = "would reformat file.py."
expected_text = "reformat"
expected_strip_text = "file.py."

# pylint: disable-next=protected-access
return_text = black_runner._removeprefix(text=expected_text, prefix=black_runner.WOULD_REFORMAT_MSG)
# pylint: disable-next=protected-access
return_strip_text = black_runner._removeprefix(
text=would_reformat_message, prefix=black_runner.WOULD_REFORMAT_MSG
)

assert return_text == expected_text
assert return_strip_text == expected_strip_text

def test_black_output_parser_with_no_issues(self) -> None:
"""Tests black_output_parser function with the results of a file with no issues."""
expected_findings = python_tool_common.Findings()
Expand All @@ -85,7 +69,7 @@ def test_black_output_parser_with_issues(self) -> None:
expected_findings = python_tool_common.Findings(
[
python_tool_common.Finding(
path=pathlib.Path("file.py"),
path=pathlib.Path(" file.py"),
Copy link
Contributor

@jannowotsch jannowotsch Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wa sthis change made on purpose?

message="Should be reformatted.",
severity=python_tool_common.Severity.WARN,
tool="black",
Expand Down
Loading
Loading