Bazel CI Release Process should build and publish Bazel docker containers #1060

philwo · 2020-11-16T14:40:57Z

The current Bazel Docker containers are being built by the RBE toolchains team, but they asked us to take this on.

We should setup some tooling and a pipeline (?) to regularly automatically build current Bazel containers for all released versions.

kevingessner · 2021-01-19T16:15:50Z

@philwo Hello, any news here? This is still blocking us on smoothly upgrading Bazel past 3.5.

philwo · 2021-01-25T10:13:07Z

@kevingessner Not yet, sorry :( But this is really important to me and I'll get to it as soon as I can, promised!

EdSchouten · 2021-03-20T15:33:34Z

Any updates on this?

chasain · 2021-06-01T16:56:20Z

I'm all for automating this, but for now can we get a manually built image pushed

johanbrandhorst · 2021-07-02T03:29:21Z

Friendly bump, blocking upgrade from 3.5.0

Apparently there are no official bazel docker images for v3.5.0+ Ref: bazelbuild/continuous-integration#1060

* Update bazel and various rules and dependencies * Drop to v3.5.0 again Apparently there are no official bazel docker images for v3.5.0+ Ref: bazelbuild/continuous-integration#1060 * Gazelle * Sum * Run go mod tidy after gazelle repos update

EdSchouten · 2021-08-09T19:28:23Z

This is still an issue.

prawadhwani · 2021-10-06T10:46:35Z

Meanwhile, updating the readme with instructions to build it manually would also help!

lbhdc · 2021-10-16T20:42:20Z

I came across this issue while searching for a docker image that supported bazel 4. I didn't find any, so I made my own and thought I would share.

# Dockerfile
FROM registry.fedoraproject.org/fedora:34

RUN dnf install -y dnf-plugins-core git patch python3 && \
    dnf copr enable -y vbatts/bazel                   && \
    dnf install -y bazel4                             && \
    dnf clean all                                     && \
    rm -rf /var/cache/yum

RUN ln -s /usr/bin/python3 /usr/bin/python

WORKDIR /home/builder

ENTRYPOINT ["bazel"]

$ docker build .
...
Successfully built <container-id>
$ docker run <container-id> --version
bazel 4.2.1

QuantamHD · 2021-10-26T13:59:58Z

@philwo This is actually more critical than ever. The base image used to build the latest container is using ubuntu 16.04 which as of April 30th is now EOL. This is a security issue now.

I've also experienced CA failures due to the wildly out of date base image.

dayfine · 2021-10-30T18:14:18Z

It has been almost a year. Seems a bit long for a P1? Should it be now bumped to P0 to help prioritization?

I came here after updating rules_python to 0.5.0. It mentions Bazel 4 is now the LTS version. I suppose P0 might make sense?

ScarletFlash · 2021-11-16T18:31:29Z

Any updates here?

This is kind of a hack to upgrade bazel to 4.2.1. When a new Bazel Docker container is released we should remove this commit, and upgrade the docker container instead. See bazelbuild/continuous-integration#1060

QuantamHD · 2021-11-16T19:38:07Z

@ScarletFlash I'm not on or affiliated with the Bazel team, but I reached out to one of my contacts inside Google to help get this prioritized.

This is kind of a hack to upgrade bazel to 4.2.1. When a new Bazel Docker container is released we should remove this commit, and upgrade the docker container instead. See bazelbuild/continuous-integration#1060

philwo · 2021-11-24T16:44:14Z

I talked with @alexeagle about this last week who wanted to help me out (thanks!!). Alex, do you already have a Dockerfile that I could try?

f0rmiga · 2021-12-14T03:16:04Z

@philwo Alex asked if I could come up with a Dockerfile for this. How about #1290?

This is necessary because the new rules_go and gazelle versions require a version of bazel newer than what's in the GCR docker image pre-built by the bazel team. Apparently, they stopped building those in 2020. bazelbuild/continuous-integration#1060 Signed-off-by: Chris Koch <chrisko@google.com>

This can be a nice feature when a .tex file contains a tikz figure. This is enabled by exposing the dvisvgm binary in the defined latex texlive toolchan. dvisvgm is dependent on ghostscript as a dynamic lib, which is why it has been added. The previously referenced bazel docker image in cirrus.yaml was not actively updated by google. The last supported version was 3.5. So the docker image was changed to the newly provided image. See: bazelbuild/continuous-integration#1060 bazelbuild/continuous-integration#1401 ghostscript is built from source using rules_foreign_cc, which can't be entirely bootstrapped at this point. (It can't build autoconf) So it needs that to be installed from e.g. apt. The bazel docker container did as far as I investigated not allow to install packages in the container, I could be wrong though. Instead I created a github workflow, which depends on bazelisk. bazelisk is a downloader for bazel. You can configure it always run with a specific version. But sense cirrus.yaml uses latest, we don't state any version meaning it will always download the latest bazel. It also enables a bazel cache which "should" speed up execution in the ci pipeline.

This can be a nice feature when a .tex file contains a tikz figure. svg generation is enabled by exposing the dvisvgm binary in the defined latex texlive toolchan. dvisvgm in turn, is dependent on ghostscript as a dynamic lib, which is why it has been added. ghostscript is built from source with rules_foreign_cc. The previously referenced bazel docker image in cirrus.yaml was not actively updated by google. The last supported version was 3.5. This resulted in @platforms, needed by rules_foreign_cc, not being available. So the docker image was changed to the newly provided image. See: bazelbuild/continuous-integration#1060 bazelbuild/continuous-integration#1401 rules_foreign_cc can't be entirely bootstrapped at this point. (It can't build autoconf) So rules_foreign_cc needs autoconf to be installed from e.g. apt. The bazel docker container, did as far as I investigated, not allow to install packages in the container, I could be wrong though. Instead I created a github workflow, which depends on bazelisk. bazelisk is a downloader for bazel. You can configure it to always run with a specific version. But sense cirrus.yaml uses latest, we don't state any version meaning it will always download the latest bazel. It also enables a bazel cache which "should" speed up execution in the ci pipeline.

abhillman · 2022-09-29T02:10:12Z

Potential workaround is to use the cockroach db bazel images https://hub.docker.com/r/cockroachdb/bazel/tags

dlorenc · 2022-12-18T20:30:46Z

Hey - we have bazel images published by Chainguard here at cgr.dev/chainguard/bazel! Feel free to try them out.

meteorcloudy · 2022-12-21T07:17:58Z

I'm closing this one since bazel docker images are already published to gcr.io/bazel-public/bazel for every new Bazel version

yogurtearl · 2023-01-02T17:56:00Z

I'm closing this one since bazel docker images are already published to gcr.io/bazel-public/bazel for every new Bazel version

@meteorcloudy could/should that info be added to the bazel docs? it is not very discoverable.

meteorcloudy · 2023-01-03T07:34:15Z

@yogurtearl SGTM, sent bazelbuild/bazel#17113

We also have documentation in our release process: https://github.com/bazelbuild/continuous-integration/blob/master/docs/release-playbook.md#push-new-docker-image

Fixes bazelbuild/continuous-integration#1060 (comment) Closes #17113. PiperOrigin-RevId: 511666892 Change-Id: Ifbfb8c9b2ba82724719101a0118f1235948f6bab

This is kind of a hack to upgrade bazel to 4.2.1. When a new Bazel Docker container is released we should remove this commit, and upgrade the docker container instead. See bazelbuild/continuous-integration#1060

It's nice to be able to find the Dockerfile for any image easily, so I'm just adding the link to the "Getting Started with Bazel Docker Container" doc. Maybe it was probably me not knowing how to navigate the Container Registry website but I couldn't find the Dockerfile for the official images. In fact, it was a bit confusing because, while googling, I did find some old versions of the website that pointed to an old registry with very old images! Luckily, trying to find what was going on, I found [bazelbuild/continuous-integration issue bazelbuild#1060](bazelbuild/continuous-integration#1060) and from there, I found the Dockerfile for the official image.

It's nice to be able to find the Dockerfile for any image easily, so I'm just adding the link to the "Getting Started with Bazel Docker Container" doc. Maybe it was probably me not knowing how to navigate the Container Registry website but I couldn't find the Dockerfile for the official images. In fact, it was a bit confusing because, while googling, I did find some old versions of the website that pointed to an old registry with very old images! Luckily, trying to find what was going on, I found [bazelbuild/continuous-integration issue #1060](bazelbuild/continuous-integration#1060) and from there, I found the Dockerfile for the official image. Closes #23072. PiperOrigin-RevId: 662079926 Change-Id: I54f6e7cc0ef74540ea3330248130e96941477322

philwo added FR P1 labels Nov 16, 2020

philwo self-assigned this Nov 16, 2020

philwo mentioned this issue Nov 18, 2020

marketplace.gcr.io/google/bazel is missing recent Bazel releases bazelbuild/bazel#12367

Closed

bshi mentioned this issue Jan 29, 2021

marketplace.gcr.io/google/bazel is missing recent Bazel releases GoogleCloudPlatform/container-definitions#12037

Open

philwo mentioned this issue Feb 8, 2021

Update l.gcr.io/google/bazel:latest bazel docker image to bazel version 4.0.0 bazelbuild/bazel#12952

Closed

fejta mentioned this issue May 10, 2021

Moving cluster logic into it's own object bazelbuild/rules_k8s#649

Closed

johanbrandhorst added a commit to johanbrandhorst/bazel-mono that referenced this issue Jul 2, 2021

Drop to v3.5.0 again

b6448b0

Apparently there are no official bazel docker images for v3.5.0+ Ref: bazelbuild/continuous-integration#1060

ZacxDev mentioned this issue Aug 12, 2021

Release 4.2 - June 2021 bazelbuild/bazel#13558

Closed

9 tasks

QuantamHD mentioned this issue Nov 16, 2021

Update Bazel to 4.2.1 hdl/bazel_rules_hdl#65

Merged

msiebuhr mentioned this issue Nov 17, 2021

Release 5.0 - January 2022 bazelbuild/bazel#14013

Closed

9 tasks

f0rmiga mentioned this issue Dec 14, 2021

feat: Dockerfile for new Bazel releases #1290

Merged

alexeagle mentioned this issue Dec 17, 2021

Official Bazel Docker container has not been released since 3.5.0 bazelbuild/bazel#14413

Closed

tetsuok mentioned this issue Aug 17, 2022

Use Ubuntu 20.04 for Bazel Docker container #1401

Merged

This was referenced Aug 24, 2022

Weighted loadbalancing (client steering), WG peer config push to clients freifunkMUC/wgkex#87

Merged

README additions & build fixes freifunkMUC/wgkex#89

Merged

alexeagle mentioned this issue Dec 19, 2022

Release 6.0.0 - December 2022 bazelbuild/bazel#16159

Closed

9 tasks

meteorcloudy closed this as completed Dec 21, 2022

alexeagle mentioned this issue Dec 23, 2022

[bazel.build] Problem with /install/docker-container bazelbuild/bazel#17070

Closed

meteorcloudy mentioned this issue Jan 3, 2023

Document the option to use docker container bazelbuild/bazel#17113

Closed

hakman mentioned this issue Feb 4, 2023

Update to go 1.19.5 kubernetes-retired/etcdadm#365

Merged

lpawelcz mentioned this issue Jun 1, 2023

Bump Bazel version hdl/bazel_rules_hdl#166

Closed

jjmaestro mentioned this issue Jul 23, 2024

Add link to the official Dockerfile config bazelbuild/bazel#23072

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bazel CI Release Process should build and publish Bazel docker containers #1060

Bazel CI Release Process should build and publish Bazel docker containers #1060

philwo commented Nov 16, 2020

kevingessner commented Jan 19, 2021

philwo commented Jan 25, 2021

EdSchouten commented Mar 20, 2021

chasain commented Jun 1, 2021

johanbrandhorst commented Jul 2, 2021

EdSchouten commented Aug 9, 2021

prawadhwani commented Oct 6, 2021

lbhdc commented Oct 16, 2021

QuantamHD commented Oct 26, 2021 •

edited

Loading

dayfine commented Oct 30, 2021

ScarletFlash commented Nov 16, 2021

QuantamHD commented Nov 16, 2021

philwo commented Nov 24, 2021

f0rmiga commented Dec 14, 2021

abhillman commented Sep 29, 2022

dlorenc commented Dec 18, 2022

meteorcloudy commented Dec 21, 2022

yogurtearl commented Jan 2, 2023

meteorcloudy commented Jan 3, 2023

Bazel CI Release Process should build and publish Bazel docker containers #1060

Bazel CI Release Process should build and publish Bazel docker containers #1060

Comments

philwo commented Nov 16, 2020

kevingessner commented Jan 19, 2021

philwo commented Jan 25, 2021

EdSchouten commented Mar 20, 2021

chasain commented Jun 1, 2021

johanbrandhorst commented Jul 2, 2021

EdSchouten commented Aug 9, 2021

prawadhwani commented Oct 6, 2021

lbhdc commented Oct 16, 2021

QuantamHD commented Oct 26, 2021 • edited Loading

dayfine commented Oct 30, 2021

ScarletFlash commented Nov 16, 2021

QuantamHD commented Nov 16, 2021

philwo commented Nov 24, 2021

f0rmiga commented Dec 14, 2021

abhillman commented Sep 29, 2022

dlorenc commented Dec 18, 2022

meteorcloudy commented Dec 21, 2022

yogurtearl commented Jan 2, 2023

meteorcloudy commented Jan 3, 2023

QuantamHD commented Oct 26, 2021 •

edited

Loading