Skip to content

Commit

Permalink
[bazel] Upgrade Bazel to 5.0.0
Browse files Browse the repository at this point in the history
Summary:
This diff updates our Bazel version to 5.0.0.

I've re-worked our patches on top of the upstream 5.0.0 tag.

- `linux-sandbox` has changed a bunch (mostly in `linux-sandbox-pid1.cc`) so I had to rewrite a bunch of the patch. The main change is that upstream has added a bunch of logic from our own patch in order to support the new hermetic sandbox flag (see bazelbuild/bazel#13279). So I've cleaned things up, removed some of our code and instead called their new code. One change is that they hardlink an empty file outside of the sandbox rather than creating new files, which sounds ok. Note that we might be able to remove even more of our own patch in favor of their hermetic support but we can do that later.
- The merkle tree computation moved from `RemoteSpawnCache` to the `RemoteExecutionService` We should be able to rewrite the patch fairly easily but they've also added an (in-process) cache for those trees (see bazelbuild/bazel#13879) so it might be helping with the slowness that we were seeing before. I'm inclined to not apply the patch to start with and we can add it back if things get much slower.

The changes are on the `dbx-2022-02-25-5.0.0` branch in the bazel repo.

Here's the list of our own commits on top of upstream:
- [[ https://sourcegraph.pp.dropbox.com/bazel/-/commit/5a121a34b1a2a39530bf6cecc3892fc4509a1735?visible=2 | DBX: Helper scripts to build dbx bazel ]]
- [[ https://sourcegraph.pp.dropbox.com/bazel/-/commit/c3707dea392806b81f2892d46ede5bf54ef02527?visible=1 | DBX: Point remotejdk URLs to magic mirror ]]
- [[ https://sourcegraph.pp.dropbox.com/bazel/-/commit/dc5a85b9a1b710230f2c786fd2cede3adb29370d?visible=2 | DBX: Make sure that the java8 toolchain uses the right options ]]
- [[ https://sourcegraph.pp.dropbox.com/bazel/-/commit/497532f9878b3b68582c12766bf034a4de6cc44a?visible=6 | DBX: rootfs patch for the linux-sandbox ]]

Also see https://blog.bazel.build/2022/01/19/bazel-5.0.html

DTOOLS-1748

Test Plan:
Will run the main projects and CI and make sure that things still work.

Ran `bzl tool //dropbox/devtools/bazel_metrics/benchmarks --target //services/metaserver edit-refresh` on both this diff and master.

On 4.1.0 on master:

```
Running no-op reload 5 times...
Finished running no-op reload! The results were:
  min: 3.01s
  avg: 3.08s
  p50: 3.08s
  max: 3.21s
Running modify metaserver/static/js/modules/core/uri.ts 5 times...
Finished running modify metaserver/static/js/modules/core/uri.ts! The results were:
  min: 5.30s
  avg: 5.78s
  p50: 5.77s
  max: 6.59s
Running modify metaserver/static/css/legacy_browse.scss 5 times...
Finished running modify metaserver/static/css/legacy_browse.scss! The results were:
  min: 4.46s
  avg: 4.83s
  p50: 4.69s
  max: 5.26s
Running add file at metaserver/static/js/modules/core/devbox-benchmark-file-{}.ts 5 times...
Finished running add file at metaserver/static/js/modules/core/devbox-benchmark-file-{}.ts! The results were:
  min: 25.69s
  avg: 26.21s
  p50: 26.22s
  max: 26.89s
Running modify metaserver/static/error/maintenance.html 5 times...
Finished running modify metaserver/static/error/maintenance.html! The results were:
  min: 4.75s
  avg: 4.85s
  p50: 4.75s
  max: 5.01s
```

On 5.0.0

```
Running no-op reload 5 times...
Finished running no-op reload! The results were:
  min: 3.48s
  avg: 3.69s
  p50: 3.48s
  max: 3.90s
Running modify metaserver/static/js/modules/core/uri.ts 5 times...
Finished running modify metaserver/static/js/modules/core/uri.ts! The results were:
  min: 5.54s
  avg: 6.34s
  p50: 5.54s
  max: 8.59s
Running modify metaserver/static/css/legacy_browse.scss 5 times...
Finished running modify metaserver/static/css/legacy_browse.scss! The results were:
  min: 4.34s
  avg: 4.75s
  p50: 5.05s
  max: 5.46s
Running add file at metaserver/static/js/modules/core/devbox-benchmark-file-{}.ts 5 times...
Finished running add file at metaserver/static/js/modules/core/devbox-benchmark-file-{}.ts! The results were:
  min: 25.55s
  avg: 25.96s
  p50: 25.64s
  max: 26.71s
Running modify metaserver/static/error/maintenance.html 5 times...
Finished running modify metaserver/static/error/maintenance.html! The results were:
  min: 4.79s
  avg: 5.33s
  p50: 5.15s
  max: 5.84s
```

GitOrigin-RevId: 0f466c5a3bde9ed1157ea936bb70826b58f2fbec
  • Loading branch information
Damien DeVille authored and jhance committed Mar 23, 2022
1 parent b567504 commit 96570f9
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 6 deletions.
7 changes: 5 additions & 2 deletions build_tools/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -281,9 +281,12 @@ cc_binary(
srcs = ["exit0.c"],
)

cc_test(
# `pass` used to be a `cc_test` but starting with Bazel 5.0.0 we ran into conflicting actions issues due
# to this test target being added as a dependency to other test and non-test targets.
# See this issue for more info: see https://github.com/bazelbuild/bazel/issues/14294.
sh_test(
name = "pass",
srcs = ["exit0.c"],
srcs = ["exit0.sh"],
)

dbx_sh_binary(
Expand Down
7 changes: 5 additions & 2 deletions build_tools/BUILD.in
Original file line number Diff line number Diff line change
Expand Up @@ -209,9 +209,12 @@ cc_binary(
srcs = ["exit0.c"],
)

cc_test(
# `pass` used to be a `cc_test` but starting with Bazel 5.0.0 we ran into conflicting actions issues due
# to this test target being added as a dependency to other test and non-test targets.
# See this issue for more info: see https://github.com/bazelbuild/bazel/issues/14294.
sh_test(
name = "pass",
srcs = ["exit0.c"],
srcs = ["exit0.sh"],
)

dbx_sh_binary(
Expand Down
4 changes: 2 additions & 2 deletions build_tools/bazel/external_workspace.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ def filename_from_label(label):

DEFAULT_EXTERNAL_URLS = {
"abseil_py": ["https://github.com/abseil/abseil-py/archive/pypi-v0.7.1.tar.gz"],
"bazel_skylib": ["https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.2/bazel-skylib-1.0.2.tar.gz"],
"bazel_skylib": ["https://github.com/bazelbuild/bazel-skylib/releases/download/1.2.0/bazel-skylib-1.2.0.tar.gz"],
"com_github_plougher_squashfs_tools": ["https://github.com/plougher/squashfs-tools/archive/4.4.tar.gz"],
"cpython_38": ["https://www.python.org/ftp/python/3.8.1/Python-3.8.1.tar.xz"],
"cpython_39": ["https://www.python.org/ftp/python/3.9.9/Python-3.9.9.tar.xz"],
Expand Down Expand Up @@ -82,7 +82,7 @@ def drte_deps(urls = DEFAULT_EXTERNAL_URLS):
http_archive(
name = "bazel_skylib",
urls = urls["bazel_skylib"],
sha256 = "97e70364e9249702246c0e9444bccdc4b847bed1eb03c5a3ece4f83dfe6abc44",
sha256 = "af87959afe497dc8dfd4c6cb66e1279cb98ccc84284619ebfec27d9c09a903de",
)

pypi_core_deps(urls)
Expand Down

0 comments on commit 96570f9

Please sign in to comment.