Release 4.0 - December 2020 #12455

philwo · 2020-11-11T09:55:13Z

Status of Bazel 4.0

Target baseline: 2020-11-12
Expected release date: 2021-01-18
List of release blockers

To report a release-blocking bug, please file a bug using the Release blocker label, and cc me.

Task list:

The text was updated successfully, but these errors were encountered:

dws · 2020-11-13T00:49:34Z

If this release includes breaking change e379ece, then please ensure that the release has a relnote for it.
I became aware of this change because it broke an earlier version of a bazel-skylib PR that I've been working on.

philwo · 2020-11-18T18:26:22Z

Bazel 4.0.0rc1 created and pushed via:

scripts/release/release.sh create 4.0.0 cd3480e4d53160d20c634134d3917c4e53ac1c49
scripts/release/release.sh push

philwo · 2020-11-18T18:32:19Z

aaaaand it failed:

ERROR: /workdir/scripts/packages/BUILD:122:21: no such package 'scripts/fish': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
/workdir/scripts/fish and referenced by '//scripts/packages:with-jdk/install.sh'

I've seen this recently when building / testing Bazel locally and I think it's just a dangling reference somewhere in our BUILD files that we didn't clean up. Investigating.

philwo · 2020-11-18T18:45:12Z

git bisect says, this is the külprit: 20febac

philwo · 2020-11-18T19:36:47Z

scripts/release/release.sh create --force_rc=2 4.0.0 37a429ad12b4c9e6a62dbae4881a1ff03b81ab40
scripts/release/release.sh push

philwo · 2020-11-18T22:20:24Z

Release builds succeeded on:

Linux x86_64
macOS x86_64
Windows x86_64
Linux arm64

Next step pushing to GCS.

philwo · 2020-11-18T22:31:18Z

Pushed and announcement for 4.0.0rc2 sent to bazel-discuss! 🎉

philwo · 2020-11-23T15:03:30Z

List of proposed cherry-picks for 4.0.0rc3:

Support Scala3 .tasty files #12529

meisterT · 2020-11-27T15:19:36Z

Just to make you aware, #12574 is still under investigation but may make another cherry pick necessary.

comius · 2020-11-30T11:48:55Z

Just to make you aware, #12574 is still under investigation but may make another cherry pick necessary.

Java_tools 10.4 is already in baseline, so no cherry pick is necessary.

wrowe · 2020-12-02T15:01:10Z

@philwo Can this be added to the list of hard showstoppers? #12545 is a regression that's been observed only since 3.7.0 release.

meisterT · 2020-12-03T15:26:55Z

#12545 is fixed by a689d67

jin · 2020-12-04T03:45:22Z

Jacoco crash reported in #12619 for rc2

limdor · 2020-12-04T09:48:15Z

@philwo in the context of LTS, is it already know what will happen with the bug fixes that should go to the next 4.x release without having to wait for 5.0? I would like to request a cherry-pick for 4.x LTS, I do not think is needed for release 4.0 because has been broken for a long time.

comius · 2020-12-07T08:46:18Z

Jacoco crash reported in #12619 for rc2

The crash was fixed by commit d90ec67, which may be cherry-picked .

comius · 2020-12-07T08:50:49Z

Another issue was discovered in Turbine, which affects Java 15 compilations (potentially also other versions) #12605. The fix is more tricky, because we are already at java_tools v11.0, and I need to backport a fix to java_tools v10 and release v10.5. (and v11.1).

Gormo · 2020-12-07T13:28:57Z

We've seen an issue on Windows on 4.0.0rc2
ERROR: C:/####/BUILD.bazel:7:8: Executing genrule @##//####:generate_ids failed: Exec failed due to IOException: Cannot delete path '/dev/null': ERROR: src/main/native/windows/file-jni.cc(156): nativeDeletePath(\?\dev\null): ERROR: src/main/native/windows/file.cc(624): DeletePath(\?\dev\null): expected an absolute Windows path

Seen both triggered from a genrule and a regular rule.

We're still unsure exactly what is triggering this issue but it's currently stopping our migration to 4.0.0. Investigation ongoing but wanted to give you a heads up.

comius · 2020-12-07T13:31:47Z

Another issue was discovered in Turbine, which affects Java 15 compilations (potentially also other versions) #12605. The fix is more tricky, because we are already at java_tools v11.0, and I need to backport a fix to java_tools v10 and release v10.5. (and v11.1).

v10.5 release is in 4e5d0fd, which should also be cherry-picked
optionally also cherrypick 9a3552c, that is turbine_deploy.jar, it doesn't affect bazel binaries, but it might help the posterity

philwo · 2020-12-07T13:40:20Z

@Gormo Thanks for pointing that out. I checked with @meteorcloudy but it doesn't ring a bell.. we'll have to investigate further. If you find out anything, please let us know!

In the meantime I'll go ahead with 4.0.0rc3 so that the new cherry-picks get some exposure.

jmillikin · 2020-12-07T13:50:55Z

Would it be possible to cherry-pick 8555789 into v4.0 ? If not, could it be queued up for the first v4.x patch release?

philwo · 2020-12-07T13:57:01Z

@jmillikin I'm currently not picky when it comes to cherry picks. 😏 Happy to add it.

philsc · 2021-01-20T05:40:51Z

I'm guessing it's too late to cherry-pick the @platforms update (e03cb63)? The one packaged in rc10 is over a year old at this point.

meisterT · 2021-01-21T07:33:15Z

@philsc that was too late indeed - we may want to consider this for the next release of the 4.x release track.

meisterT · 2021-01-21T08:09:52Z

We have released: https://github.com/bazelbuild/bazel/releases/tag/4.0.0 🥳

@vbatts @petemounce @excitoon please update your respective packages

limdor · 2021-01-21T12:19:29Z

Cool! Regarding the release bazel toolchains for 4.0, is it automatic or someone has to trigger it?
https://buildkite.com/bazel/bazel-toolchains

philwo · 2021-01-21T13:37:49Z

@smukherj1 Do we have to do anything to get bazel-toolchains 4.0 or is the process automated? 😊

petemounce · 2021-01-21T14:00:22Z

Published to chocolatey

smukherj1 · 2021-01-21T14:09:32Z

@smukherj1 Do we have to do anything to get bazel-toolchains 4.0 or is the process automated? 😊

Some manual work involved. I'll take a look

smukherj1 · 2021-01-21T19:10:49Z

@smukherj1 Do we have to do anything to get bazel-toolchains 4.0 or is the process automated? 😊

Some manual work involved. I'll take a look

@philwo unfortunately looks like this'll be delayed. #8380 broke the config generation pipeline and from the error it's not immediately obvious where the invalid escape sequence is coming from. Going to have to trace through a bunch of dependencies to figure it out :(

vbatts · 2021-01-25T19:48:02Z

i've kicked off the fedora/centos build https://copr.fedorainfracloud.org/coprs/build/1902291

…

On Thu, Jan 21, 2021 at 2:11 PM Suvanjan Mukherjee ***@***.***> wrote: @smukherj1 <https://github.com/smukherj1> Do we have to do anything to get bazel-toolchains 4.0 or is the process automated? 😊 Some manual work involved. I'll take a look @philwo <https://github.com/philwo> unfortunately looks like this'll be delayed. #8380 <#8380> broke the config generation pipeline and from the error it's not immediately obvious where the invalid escape sequence is coming from. Going to have to trace through a bunch of dependencies to figure it out :( — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12455 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAQL2LSTWCIKE37SIGHD43S3B34XANCNFSM4TRY7ORQ> .

vbatts · 2021-01-25T20:32:31Z

while the packages are still building for the actively released versions of fedora/centos, meanwhile the "rawhide" (devel branch) builds have failed on x86_64 and aarch64.
in the build log https://download.copr.fedorainfracloud.org/results/vbatts/bazel/fedora-rawhide-aarch64/01902291-bazel4/builder-live.log.gz, it seems to be due to a newer gcc/glibc:

�[1A�[K�[31m�[1mERROR: �[0m/builddir/build/BUILD/bazel-4.0.0/third_party/ijar/BUILD:47:11: Compiling third_party/ijar/zlib_client.cc failed: (Exit 1): gcc failed: error executing command 
  (cd /var/tmp/bazel_9Cijshu1/out/execroot/io_bazel && \
  exec env - \
    PATH=/builddir/build/BUILD/bazel-4.0.0/bin-hack:/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin \
    PWD=/proc/self/cwd \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/aarch64-opt/bin/third_party/ijar/_objs/zlib_client/zlib_client.d '-frandom-seed=bazel-out/aarch64-opt/bin/third_party/ijar/_objs/zlib_client/zlib_client.o' -iquote . -iquote bazel-out/aarch64-opt/bin -isystem third_party/zlib -isystem bazel-out/aarch64-opt/bin/third_party/zlib -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c third_party/ijar/zlib_client.cc -o bazel-out/aarch64-opt/bin/third_party/ijar/_objs/zlib_client/zlib_client.o)
Execution platform: //:default_host_platform
�[32m[373 / 1,085]�[0m 3 actions running
    Compiling src/google/protobuf/compiler/cpp/cpp_message.cc; 16s local
    Compiling src/google/protobuf/compiler/command_line_interface.cc; 7s local
    @com_google_protobuf//:protoc_lib; 2s local
�[1A�[KIn file included from third_party/ijar/zlib_client.cc:20:
./third_party/ijar/zlib_client.h:64:46: error: 'numeric_limits' is not a member of 'std'
   64 |   static const size_t MAX_BUFFER_SIZE = std::numeric_limits<int32_t>::max();
      |                                              ^~~~~~~~~~~~~~
./third_party/ijar/zlib_client.h:64:68: error: expected primary-expression before '>' token
   64 |   static const size_t MAX_BUFFER_SIZE = std::numeric_limits<int32_t>::max();
      |                                                                    ^
./third_party/ijar/zlib_client.h:64:71: error: '::max' has not been declared; did you mean 'std::max'?
   64 |   static const size_t MAX_BUFFER_SIZE = std::numeric_limits<int32_t>::max();
      |                                                                       ^~~
      |                                                                       std::max
In file included from /usr/lib/gcc/aarch64-redhat-linux/11/../../../../include/c++/11/algorithm:62,
                 from third_party/ijar/zlib_client.cc:16:
/usr/lib/gcc/aarch64-redhat-linux/11/../../../../include/c++/11/bits/stl_algo.h:3467:5: note: 'std::max' declared here
 3467 |     max(initializer_list<_Tp> __l, _Compare __comp)
      |     ^~~
�[32m[373 / 1,085]�[0m 3 actions running
    Compiling src/google/protobuf/compiler/cpp/cpp_message.cc; 16s local
    Compiling src/google/protobuf/compiler/command_line_interface.cc; 7s local
    @com_google_protobuf//:protoc_lib; 2s local
�[1A�[KTarget //src:bazel_nojdk failed to build
�[32m[376 / 1,085]�[0m checking cached actions
�[1A�[K�[32mINFO: �[0mElapsed time: 409.904s, Critical Path: 28.63s
�[32m[376 / 1,085]�[0m checking cached actions
�[1A�[K�[32mINFO: �[0m376 processes: 15 internal, 361 local.
�[32m[376 / 1,085]�[0m checking cached actions
�[1A�[K�[31m�[1mFAILED:�[0m Build did NOT complete successfully
�[1A�[K�[31m�[1mFAILED:�[0m Build did NOT complete successfully

using package versions:

glibc-2.32.9000-26.fc34.aarch64
libgcc-11.0.0-0.16.fc34.aarch64
libstdc++-11.0.0-0.16.fc34.aarch64

limdor · 2021-01-25T20:37:10Z

Related to #12702

Bazel 4 is going to be a long term support release. Latest version in NixPkgs so far was 3.3.1 There's a need for more recent version NixOS#97497 All versions from 3.5.0 to 3.7.1 had some reproducibility issues as noted in issue above, but there also seems to be a working PR for 3.7.1 now at NixOS#105439 Notable changes from bazel_3 setup: - put python to default bash path For autodetecting python toolchain with strict action_env on and without this change bazel would fail to autodetect host python. There are some repos that define hermetic python toolchains, but they aren't easy to use yet. Also telling python paths to bazel isn't a 1-liner it seems: - action_env=PATH would affect cache - declaring toolchain via BUILD&WORKSPACE files is not per-user but more like per-repo and affects cache too Using python from nixpkgs shouldn't be too bad in the lack of simpler hermetic python toolchain options - bazel_4.updater is bazel on `bazel query` to support new constructs in WORKSPACE (load of vars, transitive load etc). This is more robust but requires bazel to run the updater, using bazel_3 for now. This is only needed to bump package version, doesn't introduce bazel_4 build dependency on bazel_3 https://blog.bazel.build/2020/11/10/bazel-4.0-announce.html https://blog.bazel.build/2020/11/10/long-term-support-release.html bazelbuild/bazel#12455 https://github.com/bazelbuild/bazel/releases/tag/4.0.0 https://blog.bazel.build/2021/01/19/bazel-4-0.html

keith · 2021-01-28T21:37:19Z

If there's a 4.0.1 it should probably include #12882

Bazel 4 is going to be a long term support release. Latest version in NixPkgs so far was 3.3.1 There's a need for more recent version NixOS#97497 All versions from 3.5.0 to 3.7.1 had some reproducibility issues as noted in issue above, but there also seems to be a working PR for 3.7.1 now at NixOS#105439 Notable changes from bazel_3 setup: - put python to default bash path For autodetecting python toolchain with strict action_env on and without this change bazel would fail to autodetect host python. There are some repos that define hermetic python toolchains, but they aren't easy to use yet. Also telling python paths to bazel isn't a 1-liner it seems: - action_env=PATH would affect cache - declaring toolchain via BUILD&WORKSPACE files is not per-user but more like per-repo and affects cache too Using python from nixpkgs shouldn't be too bad in the lack of simpler hermetic python toolchain options - bazel_4.updater is bazel on `bazel query` to support new constructs in WORKSPACE (load of vars, transitive load etc). This is more robust but requires bazel to run the updater, using bazel_3 for now. This is only needed to bump package version, doesn't introduce bazel_4 build dependency on bazel_3 https://blog.bazel.build/2020/11/10/bazel-4.0-announce.html https://blog.bazel.build/2020/11/10/long-term-support-release.html bazelbuild/bazel#12455 https://github.com/bazelbuild/bazel/releases/tag/4.0.0 https://blog.bazel.build/2021/01/19/bazel-4-0.html

dmivankov · 2021-02-14T09:53:48Z

#12983 on --incompatible_struct_has_no_methods being potentially flipped in 4.0.0

chenrui333 · 2021-02-18T16:47:55Z

If there's a 4.0.1 it should probably include #12882

any timeline of releasing #12882 soon (this in fact makes the buildifier and buildozer PRs got disabled in the homebrew side)

philwo · 2021-02-18T17:07:21Z

Xcode 11.2 and macOS 10.14 are not supported and tested by us. We can probably cherry-pick this in the next patch release, but I wonder - isn't there a way in Bazel to disable the problematic CFLAGS flag manually if you want to build on an older platform like this? (@keith @oquenchil ?)

keith · 2021-02-18T17:09:03Z

in this specific case folks can disable this feature with --features=-debug_prefix_map_pwd_is_dot

scele · 2021-02-18T18:33:42Z

FYI - #12927 is preventing our upgrade to 4.0. If there is a 4.0.x, would be nice to get it fixed there, too.

philwo · 2021-02-24T14:24:33Z

Hi everyone,

we're tracking the next minor release Bazel 4.1 that will incorporate requested bug fixes in this issue: #13099

I've copied over all requested cherry-picks into the new issue - please check whether I forgot any and please comment there if you would like to request any other.

philwo self-assigned this Nov 11, 2020

philwo pinned this issue Nov 11, 2020

philwo added release team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website type: process labels Nov 11, 2020

limdor mentioned this issue Nov 18, 2020

Official bazel build support dart-lang/sdk#38581

Open

UebelAndre mentioned this issue Nov 19, 2020

Allow specifying a target for a given rule bazelbuild/rules_rust#489

Closed

danieljanes mentioned this issue Nov 22, 2020

Bazel's Python stub is broken on Ubuntu 20.04 LTS #11554

Closed

gregestren mentioned this issue Dec 4, 2020

Please implement output format 'graph' for cquery and aquery #10843

Closed

Wyverald mentioned this issue Jan 22, 2021

BazelCI Failing for bazel-toolchains with release Bazel bazelbuild/bazel-toolchains#927

Open

This was referenced Jan 29, 2021

4.0.0: StackOverflowError in error-prone #12926

Closed

4.0.0: git_repository patch_cmds can no longer inspect git references #12953

Closed

philwo added the P1 I'll work on this now. (Assignee required) label Feb 8, 2021

keith mentioned this issue Feb 18, 2021

Revert "Switch to -fdebug-compilation-dir" #12882

Closed

philwo mentioned this issue Feb 24, 2021

Release 4.1 - May 2021 #13099

Closed

9 tasks

philwo closed this as completed Feb 24, 2021

philwo unpinned this issue Feb 24, 2021

Release 4.0 - December 2020 #12455

Release 4.0 - December 2020 #12455

Comments

philwo commented Nov 11, 2020 • edited by meisterT Loading

Status of Bazel 4.0

dws commented Nov 13, 2020

philwo commented Nov 18, 2020

philwo commented Nov 18, 2020

philwo commented Nov 18, 2020

philwo commented Nov 18, 2020

philwo commented Nov 18, 2020

philwo commented Nov 18, 2020

philwo commented Nov 23, 2020

meisterT commented Nov 27, 2020

comius commented Nov 30, 2020

wrowe commented Dec 2, 2020

meisterT commented Dec 3, 2020

jin commented Dec 4, 2020

limdor commented Dec 4, 2020

comius commented Dec 7, 2020 • edited Loading

comius commented Dec 7, 2020 • edited Loading

Gormo commented Dec 7, 2020

comius commented Dec 7, 2020

philwo commented Dec 7, 2020

jmillikin commented Dec 7, 2020

philwo commented Dec 7, 2020

philsc commented Jan 20, 2021

meisterT commented Jan 21, 2021

meisterT commented Jan 21, 2021

limdor commented Jan 21, 2021

philwo commented Jan 21, 2021

petemounce commented Jan 21, 2021

smukherj1 commented Jan 21, 2021

smukherj1 commented Jan 21, 2021

vbatts commented Jan 25, 2021 via email

vbatts commented Jan 25, 2021

limdor commented Jan 25, 2021

keith commented Jan 28, 2021

dmivankov commented Feb 14, 2021

chenrui333 commented Feb 18, 2021

philwo commented Feb 18, 2021

keith commented Feb 18, 2021

scele commented Feb 18, 2021

philwo commented Feb 24, 2021

philwo commented Nov 11, 2020 •

edited by meisterT

Loading

comius commented Dec 7, 2020 •

edited

Loading

comius commented Dec 7, 2020 •

edited

Loading