Bazel coverage report file is empty when loading py_binary with Python version specified

[tanan]

🐞 bug report

Affected Rule

The issue is caused by the py_binary and py_test when running bazel coverage.

Is this a regression?

No.

Description

When I load py_binary and py_test with Python version specified like below and run bazel coverage command, a coverage report file is empty.

load("@python//3.10:defs.bzl", "py_binary", "py_test")

When I load these rules with the default Python version, the correct coverage report is created.

load("@rules_python//python:defs.bzl", "py_binary", "py_test")

🔬 Minimal Reproduction

Here is an example repository.
https://github.com/tanan/bazel-python-coverage/tree/python-coverage-empty

I run the command:

bazel coverage --combined_report=lcov //apps/demo/...

On the other hand, bazel coverage creates a correct file when I load the default version of py_binary and py_test. The sample repository is here:
https://github.com/tanan/bazel-python-coverage/tree/python-coverage-success

The difference between the two codes is the URL below.
tanan/bazel-python-coverage@python-coverage-success...python-coverage-empty

🔥 Exception or Error

coverage file is empty with the log WARNING: There was no coverage found.

$ bazel coverage --combined_report=lcov //apps/demo/...
Starting local Bazel server and connecting to it...
INFO: Using default value for --instrumentation_filter: "^//apps/demo[/:]".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed 3 targets (54 packages loaded, 3852 targets configured).
INFO: Found 2 targets and 1 test target...
INFO: LCOV coverage report is located at /home/foo/.cache/bazel/_bazel_foo/eb400fbb2bc2347b07965f1284d95f71/execroot/__main__/bazel-out/_coverage/_coverage_report.dat
 and execpath is bazel-out/_coverage/_coverage_report.dat
INFO: From Coverage report generation:
Dec 08, 2023 6:12:14 PM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: Found 1 tracefiles.
Dec 08, 2023 6:12:14 PM com.google.devtools.coverageoutputgenerator.Main parseFilesSequentially
INFO: Parsing file bazel-out/k8-fastbuild-ST-eff9b1c97e51/testlogs/apps/demo/lib_test/coverage.dat
Dec 08, 2023 6:12:14 PM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
Dec 08, 2023 6:12:14 PM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
Dec 08, 2023 6:12:14 PM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
Dec 08, 2023 6:12:14 PM com.google.devtools.coverageoutputgenerator.Main runWithArgs
WARNING: There was no coverage found.
INFO: Elapsed time: 27.915s, Critical Path: 8.73s
INFO: 25 processes: 19 internal, 5 linux-sandbox, 1 worker.
INFO: Build completed successfully, 25 total actions
//apps/demo:lib_test                                                     PASSED in 7.1s

Executed 1 out of 1 test: 1 test passes.

🌍 Your Environment

Operating System:

  
Ubuntu 20.04.6 LTS
  

Output of bazel version:

  
Bazelisk version: v1.11.0
Build label: 6.3.2
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Aug 8 15:48:33 2023 (1691509713)
Build timestamp: 1691509713
Build timestamp as int: 1691509713
  

Rules_python version:

  
0.27.1
  

Anything else relevant?

[tanan]

It seems that lack of the attribute of lcov_merger when running _transition_py_test have caused of this issue.

And when I debug bazel server, there are no attribute of lcov_merger in _transition_py_test. That's why the result of coverage report is empty despite successful test.
https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/analysis/test/TestActionBuilder.java#L286

if (ruleContext.isAttrDefined(":lcov_merger", LABEL)) {
  lcovMergerAttr = ":lcov_merger";
} else if (ruleContext.isAttrDefined("$lcov_merger", LABEL)) {
  lcovMergerAttr = "$lcov_merger";
}

I tried adding _lcov_merger attribute in rules_python/python/config_settings/transition.bzl and coverage command worked well.

_PY_TEST_ATTRS = {
    "_lcov_merger": attr.label(
        default = configuration_field(fragment = "coverage", name = "output_generator"),
        executable = True,
        cfg = "exec",
    ),
}

_transition_py_test = rule(
    _transition_py_impl,
    attrs = _COMMON_ATTRS | _PY_TEST_ATTRS,
    cfg = _transition_python_version,
    test = True,
)

I use the following bazel builtin code as a reference.
https://github.com/bazelbuild/bazel/blob/master/src/main/starlark/builtins_bzl/common/python/py_test_bazel.bzl#L25

Is there any thoughts on this?

[rickeylev]

Great analysis. Thanks! Yeah, adding those attributes is probably whats missing. Maybe. There's a few more attributes mentioned in that Java code, but it's all undocumented. If adding those attributes with the coverage command gives the right output, then great.