Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build: automate building and pushing of new build/builder* images #56987

Closed
jlinder opened this issue Nov 21, 2020 · 3 comments
Closed

build: automate building and pushing of new build/builder* images #56987

jlinder opened this issue Nov 21, 2020 · 3 comments
Assignees
@rickystewart
Labels
A-build-system C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@jlinder
Copy link
Collaborator

jlinder commented Nov 21, 2020

The current process for building and pushing a new builder image (build/builder*) requires the engineer making the changes to manually push the image to DockerHub. Instead, we want to put that capability into a TeamCity build configuration.

The workflow for this might look like:

  • Develop the changes to the build/builder/Dockerfile locally
  • Once satisfied with the changes, make a PR with the changes
  • Go to the new TeamCity build configuration and run it for the PR. It will output the new builder image tag.
  • Add the new builder image tag to the PR and link to the TeamCity run where it was pushed.
  • Follow the rest of the steps in rolling out the new image.
@jlinder jlinder added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-build-system labels Nov 21, 2020
@rickystewart rickystewart self-assigned this Dec 7, 2020
@rickystewart rickystewart mentioned this issue Dec 9, 2020
Merged
rickystewart added a commit to rickystewart/cockroach that referenced this issue Dec 14, 2020
@rickystewart
build: install pass, gnupg2 in teamcity agents
9866afe 
It's necessary to `docker login` our TeamCity build agents as a service
account so they can push new `build/builder` images (see cockroachdb#56987), but in
the absence of this change, attempting to do that fails with an error
like this:

```
Error saving credentials: error storing credentials - err: exit status
1, out: Cannot autolaunch D-Bus without X11 $DISPLAY
```

This is apparently because the `secretservice` binary has an X11
dependency, and `docker login` on Linux [falls back on `secretservice`](
https://docs.docker.com/engine/reference/commandline/login/#credentials-store)
if it can't find the `pass` binary.

To address this, just install the `pass` binary, which doesn't depend on
X11.

Release note: None
craig bot pushed a commit that referenced this issue Dec 15, 2020
@rickystewart
Merge #57747
1c18778 
57747: build: install pass, gnupg2 in teamcity agents r=kenliu a=rickystewart

It's necessary to `docker login` our TeamCity build agents as a service
account so they can push new `build/builder` images (see #56987), but in
the absence of this change, attempting to do that fails with an error
like this:

```
Error saving credentials: error storing credentials - err: exit status
1, out: Cannot autolaunch D-Bus without X11 $DISPLAY
```

This is apparently because the `secretservice` binary has an X11
dependency, and `docker login` on Linux [falls back on `secretservice`](
https://docs.docker.com/engine/reference/commandline/login/#credentials-store)
if it can't find the `pass` binary.

To address this, just install the `pass` binary, which doesn't depend on
X11.

Release note: None

Co-authored-by: Ricky Stewart <ricky@cockroachlabs.com>
@rickystewart
Copy link
Collaborator

This is implemented as Internal / Release / Build and Push New Builder Image.

@jlinder
Copy link
Collaborator Author

jlinder commented Jan 26, 2021 •
edited
Loading

I noticed I forgot a step for updating the documentation. Can you also update the related docs here to reflect the new workflow?

And I'm curious, what is the workflow you envisioned with this implementation? I imagine it would be something like this:

  1. Develop and test the changes locally
  2. Push them to a PR
  3. Get review of the PR (minus the new version number, which gets added later)
  4. Ask someone with access to use the new build configuration to build and push the image using the PR's branch (people with access would currently be any developer; in the future this may be just dev inf folks who can also review the PR before running the build config; the access part don't necessarily need to be added to the docs right now)
  5. Update the PR with the new version tag in build/builder.sh

rickystewart added a commit to rickystewart/cockroach that referenced this issue Feb 5, 2021
@rickystewart
build: fix documentation on updating the builder image given cockroac…
4c1976b 
…hdb#56987

Release note: None
@rickystewart
Copy link
Collaborator

Yes, that's essentially the shape of the workflow that I imagined. I've submitted a pull request to update the docs accordingly (#59861).

rickystewart added a commit to rickystewart/cockroach that referenced this issue Feb 11, 2021
@rickystewart
build: fix documentation on updating the builder image given cockroac…
5515393 
…hdb#56987

Release note: None
rickystewart added a commit to rickystewart/cockroach that referenced this issue Feb 12, 2021
@rickystewart
build: fix documentation on updating the builder image given cockroac…
aa05da8 
…hdb#56987

Release note: None
craig bot pushed a commit that referenced this issue Feb 16, 2021
@rickystewart
Merge #59861
45db472 
59861: build: fix documentation on updating the builder image given #56987 r=rickystewart a=rickystewart

Release note: None

Co-authored-by: Ricky Stewart <ricky@cockroachlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rickystewart rickystewart

Labels
A-build-system C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jlinder @rickystewart