Protobuf is broken with Bazel 6

[sgowroji]

Error in building with Bazel 6.1.2. Could you please upgrade to the latest bazel version.

protobuf/.github/workflows/test_cpp.yml

Line 36 in e5679c0

- config: { name: "Bazel4" }

Protobuf: :ubuntu: 20.04 LTS (OpenJDK 11, gcc 9.4.0), :darwin: (OpenJDK 11, Xcode)

CC : Bazel team @meteorcloudy

[meteorcloudy]

When building with Bazel 6.1.2, we got the following error message:

(00:15:16) ERROR: /workdir/java/kotlin-lite/BUILD.bazel:119:24: in java_lite_proto_library rule //java/kotlin-lite:evil_names_proto3_java_proto_lite:
Traceback (most recent call last):
	File "/virtual_builtins_bzl/common/java/proto/java_lite_proto_library.bzl", line 105, column 42, in _rule_impl
Error: <target @com_google_protobuf//java/core:lite> (rule '_osgi_jar') doesn't contain declared provider 'ProguardSpecProvider'

[haberman]

Protobuf is broken by their own changes.

Could you include more details about what is broken? Are we talking about the Bazel 6.1.2 breakage shown above by @meteorcloudy , or is there something else also?

[meteorcloudy]

@haberman I think it's a combination of both.
Bazel 6.1.2 + protobuf@a commit after 5d3d9aa
Check the build history here: https://buildkite.com/bazel/protobuf/builds?branch=main

[haberman]

Protobuf has not yet tested or advertised compatibility with Bazel 6. So it is not too surprising that there could be incompatibilities with Bazel 6.

But the original bug report said "Protobuf is broken by their own changes." Is there a place where Protobuf itself is using Bazel 6? Or is it merely third parties who are assuming Bazel 6 compatibility where none has been promised?

Bazel 6.1.2 + protobuf@a commit after 5d3d9aa

Are you saying that 5d3d9aa is the commit that broke Bazel 6 compatibility? That would be surprising, as that commit does not touch any BUILD/.bzl files

[meteorcloudy]

@haberman Sorry, to give a bit of context, we are from the Bazel team. Since protobuf is a pretty important dependency, we maintain a build for protobuf on Bazel CI, the CI config is here, and the pipeline is https://buildkite.com/bazel/protobuf/builds/4515. This pipeline just by default uses the latest Bazel release to build and test protobuf. However, looks like protobuf hasn't fully migrated to the latest Bazel release.

Are you saying that 5d3d9aa is the commit that broke Bazel 6 compatibility?

No, that commit is basically the last green protobuf commit on Bazel CI, the breaking commit should happen between (5d3d9aa, e56cf5d ]
last green: https://buildkite.com/bazel/protobuf/builds/4471
first red: https://buildkite.com/bazel/protobuf/builds/4472

We can turn the build green by adjusting the build config file we use for protobuf on our CI, but eventually this needs to be fixed by protobuf when you adopt new Bazel version.

[meteorcloudy]

@haberman BTW, what would be the minimal targets you would recommend we should specify in protobuf.yml to verify the basic functionalities of protobuf?

[haberman]

Thanks for the clarifications. Based on what you wrote, it sounds like the title of this bug should be "Build is broken with Bazel 6." Would you agree?

The references to Bazel 4 seem unrelated. We have compatibility tests that exercise Bazel 4, but these are not causing the problem. The problem is just that we aren't compatible with Bazel 6 yet.

[meteorcloudy]

Agree, /cc @sgowroji can you please update the issue title and description to summarize the issue accurately?

[sgowroji]

Sorry for the confusion, It's updated now. Thanks !

[haberman]

@haberman BTW, what would be the minimal targets you would recommend we should specify in protobuf.yml to verify the basic functionalities of protobuf?

I think it depends on what you count as "basic functionalities." For C++, I would recommend trying to build :protoc. But the repo has many languages in it (Java, Python, C#, Ruby, PHP, Objective-C, etc), so any of them could experience unique issues.

[meteorcloudy]

@haberman Thanks, we mostly care about C++, so "//:protoc" sounds correct.

[fowles]

The protobuf 23.0 release is already underway and we fully expect bazel 6 support to not be ready for it. Once that is out the door, we can visit doing this at a reasonable pace.

[davido]

@meteorcloudy I'm getting this failure first, when trying to build with bazel 6.1.2 on protobuf@HEAD (763d852)

  $ bazeldev build :protoc
ERROR: Traceback (most recent call last):
	File "/home/davido/projects/protobuf/WORKSPACE", line 65, column 20, in <toplevel>
		kotlin_repositories()
	File "/home/davido/.cache/bazel/_bazel_davido/79cbbbad7f51559e6c3ea804ad112d72/external/io_bazel_rules_kotlin/src/main/starlark/core/repositories/initialize.bzl", line 96, column 21, in kotlin_repositories
		rules_repository(
Error in repository_rule: invalid user-provided repo name '': valid names may contain only A-Z, a-z, 0-9, '-', '_', '.', and must start with a letter
ERROR: Error computing the main repository mapping: at /home/davido/.cache/bazel/_bazel_davido/79cbbbad7f51559e6c3ea804ad112d72/external/io_bazel_rules_kotlin/kotlin/core.bzl:2:5: at /home/davido/.cache/bazel/_bazel_davido/79cbbbad7f51559e6c3ea804ad112d72/external/io_bazel_rules_kotlin/kotlin/internal/opts.bzl:22:5: Encountered error while reading extension file 'jvm/opts.bzl': no such package '@io_bazel_rules_kotlin_configured//jvm': error loading package 'external': Could not load //external package

If I bump rule_kotlin to the recent release: v1.7.1 in protobuf_deps.bzl:

diff --git a/protobuf_deps.bzl b/protobuf_deps.bzl
index 7e2caa330..c9a18b413 100644
--- a/protobuf_deps.bzl
+++ b/protobuf_deps.bzl
@@ -88,6 +88,6 @@ def protobuf_deps():
     if not native.existing_rule("io_bazel_rules_kotlin"):
         http_archive(
             name = "io_bazel_rules_kotlin",
-            urls = ["https://github.com/bazelbuild/rules_kotlin/releases/download/v1.5.0-beta-4/rules_kotlin_release.tgz"],
-            sha256 = "6cbd4e5768bdfae1598662e40272729ec9ece8b7bded8f0d2c81c8ff96dc139d",
+            urls = ["https://github.com/bazelbuild/rules_kotlin/releases/download/v1.7.1/rules_kotlin_release.tgz"],
+            sha256 = "fd92a98bd8a8f0e1cdcb490b93f5acef1f1727ed992571232d33de42395ca9b3",
         )

I am getting another failure:

  $ bazeldev build :protoc
ERROR: /home/davido/.cache/bazel/_bazel_davido/79cbbbad7f51559e6c3ea804ad112d72/external/bazel_tools/platforms/BUILD:89:6: in alias rule @bazel_tools//platforms:windows: Constraints from @bazel_tools//platforms have been removed. Please use constraints from @platforms repository embedded in Bazel, or preferably declare dependency on https://github.com/bazelbuild/platforms. See https://github.com/bazelbuild/bazel/issues/8622 for details.
ERROR: /home/davido/.cache/bazel/_bazel_davido/79cbbbad7f51559e6c3ea804ad112d72/external/bazel_tools/platforms/BUILD:89:6: Analysis of target '@bazel_tools//platforms:windows' failed
ERROR: /home/davido/projects/protobuf/BUILD:513:10: While resolving toolchains for target //:protoc: invalid registered toolchain '@bazel_skylib//toolchains/unittest:cmd_toolchain': 
ERROR: Analysis of target '//:protoc' failed; build aborted: 
INFO: Elapsed time: 0.158s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

that is related to the --incompatible_use_platforms_repo_for_constraints option flip: in bazelbuild/bazel#8622

Passing --incompatible_use_platforms_repo_for_constraints=false would fix that:

 $ bazeldev build --incompatible_use_platforms_repo_for_constraints=false :protoc
INFO: Build option --incompatible_use_platforms_repo_for_constraints has changed, discarding analysis cache.
INFO: Analyzed target //:protoc (7 packages loaded, 623 targets configured).
INFO: Found 1 target...
Target //:protoc up-to-date:
  bazel-bin/protoc
INFO: Elapsed time: 78.239s, Critical Path: 13.30s
INFO: 198 processes: 8 internal, 190 linux-sandbox.
INFO: Build completed successfully, 198 total actions

UPDATE

Sorry, my bad, I was on the wrong branch, switching to the new main branch is failing with the same issue as on Buildkite:

  $ bazeldev build //conformance:conformance_java_proto_lite
Starting local Bazel server and connecting to it...
ERROR: /home/davido/projects/protobuf/conformance/BUILD.bazel:76:24: in java_lite_proto_library rule //conformance:conformance_java_proto_lite: 
Traceback (most recent call last):
	File "/virtual_builtins_bzl/common/java/proto/java_lite_proto_library.bzl", line 105, column 42, in _rule_impl
Error: <target @com_google_protobuf//java/core:lite> (rule '_osgi_jar') doesn't contain declared provider 'ProguardSpecProvider'
ERROR: /home/davido/projects/protobuf/conformance/BUILD.bazel:76:24: Analysis of target '//conformance:conformance_java_proto_lite' failed
ERROR: Analysis of target '//conformance:conformance_java_proto_lite' failed; build aborted: 
INFO: Elapsed time: 22.317s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (175 packages loaded, 1886 targets configured)

[meteorcloudy]

Error: <target @com_google_protobuf//java/core:lite> (rule '_osgi_jar') doesn't contain declared provider 'ProguardSpecProvider'

/cc @comius do you have any suggestion regarding this failure?

[meteorcloudy]

Not sure what happened, protobuf is green with Bazel 6.1.2 at HEAD: https://buildkite.com/bazel/protobuf/builds/4533

[keith]

sounds like we can close this one