Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Task to build base multi-arch image #3402

Merged
merged 1 commit into from
Oct 23, 2020

Conversation

barthy1
Copy link
Member

@barthy1 barthy1 commented Oct 16, 2020

Changes

The Task allows to build and push base pipeline multi-arch image using docker buildx extension and can be executed on x86 cluster. The result will be built and published for linux/amd64, "linux/s390x", "linux/arm64" and "linux/ppc64le" platforms in form of multi-arch image.
dind privileged sidecar is used to be able to use qemu emulation feature for buildx.
Code was tested with GKE + dogfooding env.

It is proposed to replace build-push-base-images Task and produce multi-arch gcr.io/tekton-nightly/github.com/tektoncd/pipeline/build-base:latest image instead of amd64 only.

This approach to get multi-arch base image for pipeline was suggested in another PR tektoncd/plumbing#603 and shows up as better alternative in case if it works on dogfooding infrastructure.

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you
review them:

  • Includes tests (if functionality changed/added)
  • Includes docs (if user facing)
  • Commit messages follow commit message best practices
  • Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.

Double check this list of stuff that's easy to miss:

Reviewer Notes

If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.

Release Notes

NONE

@tekton-robot tekton-robot added the release-note-none Denotes a PR that doesnt merit a release note. label Oct 16, 2020
@tekton-robot tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Oct 16, 2020
@barthy1
Copy link
Member Author

barthy1 commented Oct 16, 2020

cc @vdemeester @afrittoli
Will be appreciated for your opinion and if possible verification with dogfooding cluster

@barthy1
Copy link
Member Author

barthy1 commented Oct 17, 2020

/retest

@afrittoli
Copy link
Member

/kind misc

@tekton-robot tekton-robot added the kind/misc Categorizes issue or PR as a miscellaneuous one. label Oct 20, 2020
@afrittoli
Copy link
Member

Tested successfully in: https://dashboard.dogfooding.tekton.dev/#/namespaces/default/taskruns/build-multiarch-base-image-run-sf5mf?step=build-image with:

tkn task start build-multiarch-base-image -p imageRegistry=gcr.io/tekton-releases/staging -p pathToProject=github.com/tektoncd/pipeline -i source=tekton-pipeline-git-master -o builtBaseImage=base-image

Resulting image:
https://console.cloud.google.com/gcr/images/tekton-releases/GLOBAL/staging/github.com/tektoncd/pipeline/build-base@sha256:a440083ebb62831f87fa2580b8a1eb7f56b9f68a0bcb3abcdc44d81a6dc37ac2/details?tab=info&project=tekton-releases

Manifest:

{
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "schemaVersion": 2,
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:3f8389c74cee1d52aed2cdfa837d0ef26f0602f53a8050195d42575bb0127e8b",
         "size": 739,
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:4b666bd1a3b89dc4b357f5357ac82fe5d5d09a7e318a333f8a95df6e00a51a73",
         "size": 739,
         "platform": {
            "architecture": "s390x",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:ff0cc637d18d54f2f8321a16843b0467a3362c1b601616d83f802f097c26f6ea",
         "size": 739,
         "platform": {
            "architecture": "ppc64le",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "digest": "sha256:610fad36a99742eabc87009b8aa27d1102fab671d29816b4c0c5ce8b0b0c1d7a",
         "size": 739,
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}

@afrittoli
Copy link
Member

/cc @vdemeester

This Task allows to build base pipeline multi-arch image using docker
buildx extention and can be executed on adm64 cluster. The result will
be built and published for `linux/amd64`, "linux/s390x", "linux/arm64"
and "linux/ppc64le" platforms in form of multi-arch image.

dind privileged sidecar is used to be able to use qemu emulation feature
for buildx.

The Task is used for nightly builds

Signed-off-by: Yulia Gaponenko <yulia.gaponenko1@de.ibm.com>
@barthy1
Copy link
Member Author

barthy1 commented Oct 20, 2020

as Task itself was tested with dogfooding cluster (thanks @afrittoli !), I've updated the PR to use the task for nightly tekton builds

Copy link
Member

@vdemeester vdemeester left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/meow

@tekton-robot
Copy link
Collaborator

@vdemeester: cat image

In response to this:

/meow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tekton-robot tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 20, 2020
@barthy1
Copy link
Member Author

barthy1 commented Oct 20, 2020

/retest

Copy link
Member

@afrittoli afrittoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for this!
Since we not build the base image in a dedicated task, it would be possible to use the sha of the built image in the publish task instead of the latest tag build-base:latest- but this is something we can add in a follow-up PR.

/approve

- name: builtBaseImage
resource: builtBaseImage
- name: publish-images
runAfter: [build-base-image, build, unit-tests]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: since build-base-image already depends on [build, unit-tests], this can depend on build-base-image alone.

@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afrittoli, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [afrittoli,vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@barthy1
Copy link
Member Author

barthy1 commented Oct 23, 2020

hi @pritidesai @dibyom can somebody of you have a look at this PR, as you are in reviewers list 😊 ?

@afrittoli
Copy link
Member

/lgtm

@tekton-robot tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 23, 2020
@tekton-robot tekton-robot merged commit f37860f into tektoncd:master Oct 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/misc Categorizes issue or PR as a miscellaneuous one. lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesnt merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants