Chore: rework Bazel build system

[hcoona]

Totally rewrite the bazel build support.

1. Support latest version of bazel.

2. Build all external dependencies (except for mesalink) with bazel.

3. Support Protobuf 3.19.1 (breaking changes here: protocolbuffers/protobuf@624d29d)
   

4. Support generate compile_commands.json for bazel project.

5. Rework the options flags & move all of them into //bazel/config

Leave building with mesalink for future. I think maybe all dependencies should link against mesalink instead of openssl.

[chenzhangyi]

Nice job!

Only one question: Will the changes break the compilation of users with old versions of bazel, e.g. 0.25

[hcoona]

@chenzhangyi In short it would break building with bazel 0.25. Many dependencies relying on a higher version of bazel. I didn't verify each of them. However, rules_foreign_cc 0.7.0 requires bazel v4.0.0+ (See https://github.com/bazelbuild/rules_foreign_cc/blob/0.7.0/README.md). We build several external dependencies (openssl, libevent, thrift, ...) with it.

[hcoona]

The protobuf arena changes are still breaking. I'll fix it with GOOGLE_PROTOBUF_VERSION.

[hcoona]

Fixed protobuf compatibility & separate it into another commit.

The failing unit test seems to be unrelated with this change. The only failing unittest is

Got it. After installed curl, it passed.

[hcoona]

@wwbmmm PTAL

[wwbmmm]

@wwbmmm PTAL

LGTM

[hcoona]

Is there something else to make it merged?

[lorinlee]

Hi @hcoona , Thank you so much for your contribution! But I have a little concern, some projects may depend on bazel 0.2x, like TensorFlow 1.x, and we have seen some teams using bRPC in TensorFlow. So, maybe we should consider the compatibility issues. cc @chenzhangyi @wwbmmm

[wwbmmm]

@hcoona Can you extract the changes related to “Support Protobuf 3.19.1” into a separate PR so that this part can be merged faster？

[hcoona]

@wwbmmm Sure, split the protobuf support to #1679

[hcoona]

@lorinlee @chenzhangyi In bazel philosophy, the final project owner need to take responsibility for all of its dependencies, AKA. specify how to download & build them in WORKSPACE file. For this change, there's nothing special for the BUILD files, so it should works fine building a project depends on brpc with an earlier version of Bazel if the project owner could provide necessary indirect dependencies for bRPC. I'll verify it offline, and ping you guys later few days.

[hcoona]

Do my best to make it compiles with bazel 0.28.0 (0.27.1 failed). See example/build_with_old_bazel. @lorinlee Is this fine enough?

It means that if a user used bRPC in their project, it can compile with bazel 0.28.0 and higher.

[hcoona]

@chenzhangyi @lorinlee ping~

[mapx]

Would you help merge it?

[hcoona]

@chenzhangyi Would you help to take a look at it? The user project depending on bRPC would still compile if they are using bazel v0.28.0 and higher. example/build_with_old_bazel verified it.

[372046933]

Will it be merged soon?

[guodongxiaren]

@hcoona please reslove conflicts, thanks.

[hcoona]

@guodongxiaren PTAL

[guodongxiaren]

@zyearn @wwbmmm please review this pull request and check whether it can be merged.

[wwbmmm]

@zyearn @wwbmmm please review this pull request and check whether it can be merged.

LGTM
@zyearn please see if there is any problem to merge it.

[guodongxiaren]

@zyearn @jamesge please review

[zyearn]

What is this file used for? Can we delete it?

[hcoona]

This solved a noisy compiler warning. I suggest applying it. The final user could use their provided glog. It's harmless.

[zyearn]

Do we need to delete all the files under build_with_old_bazel?

[hcoona]

It's to prove we can build brpc with old version of bazel. It's concerned by many people above in the thread.

It's under example folder. We can delete it if we want.

[Huixxi]

It's a great contribution, and could you please share a related blog or doc for that pr? I think it will be a good learning material and we will post it to our official website, please consider this, thanks a lot.

[hcoona]

@zyearn PTAL. I removed bazel/third_party/glog/0001-improvement-copts.patch & rename bazel/third_party/glog/0002-mark-override.patch to a more meaningful name 0001-mark-override-resolve-warning.patch. If you think the example of building with old version of bazel for user is not necessary, I can remove it.

The CI failure cannot be reproduced on my dev environment with GCC 9.4.0 & bazel 4.2.2 by running CC=gcc CXX=g++ PURPOSE=compile-with-bazel-all-options sh build_in_travis_ci.sh. I looked the source code, it shouldn't be such error. Please help to rerun the failed jobs if possible.

[hcoona]

@Huixxi I didn't realize there was any special work. Please look the Bazel official document & some existing repo driven by Bazel. Please raise to me if there was truly something special with details, I can help to find some document about them.

[wwbmmm]

If you think the example of building with old version of bazel for user is not necessary, I can remove it.

I think the example of building with old version of bazel is useful. No need to remove it.

[zyearn]

CI is fixed in the latest version. Could you merge master and submit again?

[hcoona]

@zyearn It works after rebase.

[zyearn]

LGTM. Thanks for the contribution.

Btw, what do you think is the future work for improving Bazel in brpc?

[0x2Adr1]

Looks like I am coming a few minutes too late after the merge! There is a slight change to be made to make it work with Mac M1:

--- BUILD.bazel
+++ BUILD.bazel
@@ -48,7 +48,7 @@ LINKOPTS = [
     "-pthread",
     "-ldl",
 ] + select({
-    "@bazel_tools//tools/osx:darwin": [
+    "@platforms//os:macos": [
         "-framework CoreFoundation",
         "-framework CoreGraphics",
         "-framework CoreData",
@@ -206,7 +206,7 @@ BUTIL_SRCS = [
     "src/butil/recordio.cc",
     "src/butil/popen.cpp",
 ] + select({
-    "@bazel_tools//tools/osx:darwin": [
+    "@platforms//os:macos": [
         "src/butil/time/time_mac.cc",
         "src/butil/mac/scoped_mach_port.cc",
     ],
@@ -322,7 +322,7 @@ cc_library(
         "//bazel/config:brpc_with_glog": ["@com_github_google_glog//:glog"],
         "//conditions:default": [],
     }) + select({
-        "@bazel_tools//tools/osx:darwin": [":macos_lib"],
+        "@platforms//os:macos": [":macos_lib"],
         "//conditions:default": [],
     }),
 )

[zyearn]

@0x2Adr1 Could you make a PR separately?

[hcoona]

LGTM. Thanks for the contribution.

Btw, what do you think is the future work for improving Bazel in brpc?

It's good enough now. I don't think we need to adjust it unless we plan to change the project structure or Bazel bring a big change.

[guodongxiaren]

@hcoona MacOs上使用bazel似乎还有问题。烦请试试

[hcoona]

@guodongxiaren I've no macOS environment. Please help to post the error message & I'll try to investigate it. And please notice that @0x2Adr1 raise some additional fixes for macOS, please also try it.

[Huixxi]

It's better to update the getting_start document (cn/en) with a new "Compile brpc with bazel" part under each platform, but I'm not familiar with bazel...