Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build: add builder args to support build with nerdctl #289

Merged
merged 2 commits into from
Feb 22, 2023
Merged

build: add builder args to support build with nerdctl #289

merged 2 commits into from
Feb 22, 2023

Conversation

zwpaper
Copy link
Member

@zwpaper zwpaper commented Feb 16, 2023

What type of PR is this?

What this PR does / why we need it:

currently we can only use docker to build kwok in build.sh, this PR add support for nerdctl.

and updated test to drop docker dependency when testing nerdctl

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Feb 16, 2023
@netlify
Copy link

netlify bot commented Feb 16, 2023 •
edited
Loading

✅ Deploy Preview for k8s-kwok canceled.

Name Link
🔨 Latest commit 899e6b9
🔍 Latest deploy log https://app.netlify.com/sites/k8s-kwok/deploys/63f4f3499f67070008fe1107

@k8s-ci-robot
Copy link
Contributor

Hi @zwpaper. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 16, 2023
wzshiming
wzshiming reviewed Feb 16, 2023
View reviewed changes
Copy link
Member

@wzshiming wzshiming left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test? Adding a job to CI covers building images with docker and nerdctl.

images/kwok/build.sh Outdated Show resolved Hide resolved
images/kwok/build.sh Outdated Show resolved Hide resolved
images/kwok/build.sh Show resolved Hide resolved
images/kwok/build.sh Outdated Show resolved Hide resolved
@zwpaper
Copy link
Member Author

zwpaper commented Feb 16, 2023

we already have a test here https://github.com/kubernetes-sigs/kwok/pull/289/files#diff-7c08be3edc3ca1cc580db19a846f6f16c67134a2dff1f1d54e955bda77cef772R66

@wzshiming
Copy link
Member

we already have a test here https://github.com/kubernetes-sigs/kwok/pull/289/files#diff-7c08be3edc3ca1cc580db19a846f6f16c67134a2dff1f1d54e955bda77cef772R66

Add below to job.test-build in .github/workflows/test.yaml

    strategy:
      matrix:
        builder:
          - docker
          - nerdctl
    env:
      BUILDER: ${{ matrix.builder }}

@zwpaper zwpaper force-pushed the build-nerdctl-without-docker branch from 6d5db47 to d0e20e1 Compare February 16, 2023 08:59
@zwpaper
Copy link
Member Author

zwpaper commented Feb 16, 2023

ok, let me add the tests later, I will let you know when done

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 16, 2023
wzshiming
wzshiming reviewed Feb 16, 2023
View reviewed changes
images/kwok/build.sh Outdated Show resolved Hide resolved
wzshiming
wzshiming reviewed Feb 16, 2023
View reviewed changes
images/kwok/build.sh Show resolved Hide resolved
images/kwok/build.sh Show resolved Hide resolved
images/kwok/build.sh Show resolved Hide resolved
images/kwok/build.sh Show resolved Hide resolved
@zwpaper zwpaper force-pushed the build-nerdctl-without-docker branch from d0e20e1 to 12fc04b Compare February 16, 2023 16:57
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 16, 2023
@zwpaper zwpaper force-pushed the build-nerdctl-without-docker branch from 12fc04b to 85ac763 Compare February 16, 2023 17:11
@zwpaper
Copy link
Member Author

zwpaper commented Feb 16, 2023

I have also added set -eo pipefail to the 2 build.sh because it will not fail the test if errors happened in CI previously:
image

by adding set -eo pipefail, it will fail the script if any step failed.

because macOS realpath does not support --relative-to, it would failed the script, so I drop the --relative-to as it is ok to use full path of Dockerfile

@zwpaper
Copy link
Member Author

zwpaper commented Feb 16, 2023

@wzshiming test passed, we should be good to go

/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 16, 2023
@wzshiming
Copy link
Member

wzshiming commented Feb 17, 2023 •
edited
Loading

Before

$ DRY_RUN=true make build-image build-cluster-image
GOOS=linux GOARCH=arm64 go build -ldflags '-X sigs.k8s.io/kwok/pkg/consts.Version=v0.1.0-50-hash -X sigs.k8s.io/kwok/pkg/consts.KubeVersion=v1.26.0' -o ./bin/linux/arm64/kwok ./cmd/kwok
docker buildx build --tag=kwok:v0.1.0-50-hash --platform=linux/arm64 --load -f images/kwok/Dockerfile .
GOOS=linux GOARCH=arm64 go build -ldflags '-X sigs.k8s.io/kwok/pkg/consts.Version=v0.1.0-50-hash -X sigs.k8s.io/kwok/pkg/consts.KubeVersion=v1.26.0' -o ./bin/linux/arm64/kwok ./cmd/kwok
docker buildx build --tag=kwok:v0.1.0-50-hash --platform=linux/arm64 --load -f images/kwok/Dockerfile .
docker buildx build --build-arg=kube_version=v1.26.0 --tag=cluster:v0.1.0-50-hash-k8s.v1.26.0 --platform=linux/arm64 --load -f images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.25.3 --tag=cluster:v0.1.0-50-hash-k8s.v1.25.3 --platform=linux/arm64 --load -f images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.24.7 --tag=cluster:v0.1.0-50-hash-k8s.v1.24.7 --platform=linux/arm64 --load -f images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.23.13 --tag=cluster:v0.1.0-50-hash-k8s.v1.23.13 --platform=linux/arm64 --load -f images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.22.15 --tag=cluster:v0.1.0-50-hash-k8s.v1.22.15 --platform=linux/arm64 --load -f images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.21.14 --tag=cluster:v0.1.0-50-hash-k8s.v1.21.14 --platform=linux/arm64 --load -f images/cluster/Dockerfile .

After for docker

$ BUILDER=docker DRY_RUN=true make build-image build-cluster-image
GOOS=linux GOARCH=arm64 go build -ldflags '-X sigs.k8s.io/kwok/pkg/consts.Version=v0.1.0-59-hash -X sigs.k8s.io/kwok/pkg/consts.KubeVersion=v1.26.0' -o ./bin/linux/arm64/kwok ./cmd/kwok
docker buildx build --tag=kwok:v0.1.0-59-hash --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/kwok/Dockerfile .
GOOS=linux GOARCH=arm64 go build -ldflags '-X sigs.k8s.io/kwok/pkg/consts.Version=v0.1.0-59-hash -X sigs.k8s.io/kwok/pkg/consts.KubeVersion=v1.26.0' -o ./bin/linux/arm64/kwok ./cmd/kwok
docker buildx build --tag=kwok:v0.1.0-59-hash --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/kwok/Dockerfile .
docker buildx build --build-arg=kube_version=v1.26.0 --tag=cluster:v0.1.0-59-hash-k8s.v1.26.0 --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.25.3 --tag=cluster:v0.1.0-59-hash-k8s.v1.25.3 --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.24.7 --tag=cluster:v0.1.0-59-hash-k8s.v1.24.7 --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.23.13 --tag=cluster:v0.1.0-59-hash-k8s.v1.23.13 --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.22.15 --tag=cluster:v0.1.0-59-hash-k8s.v1.22.15 --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
docker buildx build --build-arg=kube_version=v1.21.14 --tag=cluster:v0.1.0-59-hash-k8s.v1.21.14 --platform=linux/arm64 --load -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .

After for nerdctl

$ BUILDER=nerdctl DRY_RUN=true make build-image build-cluster-image
GOOS=linux GOARCH=arm64 go build -ldflags '-X sigs.k8s.io/kwok/pkg/consts.Version=v0.1.0-59-hash -X sigs.k8s.io/kwok/pkg/consts.KubeVersion=v1.26.0' -o ./bin/linux/arm64/kwok ./cmd/kwok
nerdctl build --tag=kwok:v0.1.0-59-hash --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/kwok/Dockerfile .
GOOS=linux GOARCH=arm64 go build -ldflags '-X sigs.k8s.io/kwok/pkg/consts.Version=v0.1.0-59-hash -X sigs.k8s.io/kwok/pkg/consts.KubeVersion=v1.26.0' -o ./bin/linux/arm64/kwok ./cmd/kwok
nerdctl build --tag=kwok:v0.1.0-59-hash --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/kwok/Dockerfile .
nerdctl build --build-arg=kube_version=v1.26.0 --tag=cluster:v0.1.0-59-hash-k8s.v1.26.0 --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
nerdctl build --build-arg=kube_version=v1.25.3 --tag=cluster:v0.1.0-59-hash-k8s.v1.25.3 --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
nerdctl build --build-arg=kube_version=v1.24.7 --tag=cluster:v0.1.0-59-hash-k8s.v1.24.7 --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
nerdctl build --build-arg=kube_version=v1.23.13 --tag=cluster:v0.1.0-59-hash-k8s.v1.23.13 --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
nerdctl build --build-arg=kube_version=v1.22.15 --tag=cluster:v0.1.0-59-hash-k8s.v1.22.15 --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .
nerdctl build --build-arg=kube_version=v1.21.14 --tag=cluster:v0.1.0-59-hash-k8s.v1.21.14 --platform=linux/arm64 -f /Users/zsm/go/src/sigs.k8s.io/kwok/images/cluster/Dockerfile .

The modified DRY_RUN will output full path , which is not very convenient when adding tests to it.
Changes in the release process often don't show up immediately and only cause glitches in the next release, so we need to be careful, I'll add this part of the e2e first later.

wzshiming
wzshiming requested changes Feb 17, 2023
View reviewed changes
images/cluster/build.sh Outdated Show resolved Hide resolved
images/kwok/build.sh Outdated Show resolved Hide resolved
images/kwok/build.sh Outdated Show resolved Hide resolved
@wzshiming
Copy link
Member

#292 I add a test to the original releases.

@zwpaper zwpaper mentioned this pull request Feb 17, 2023
Merged
@wzshiming
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 20, 2023
@zwpaper
build: add builder args to support build with nerdctl
e484ec1 
Signed-off-by: Wei Zhang <kweizh@gmail.com>
@zwpaper zwpaper force-pushed the build-nerdctl-without-docker branch 2 times, most recently from 87f8bb0 to 94622b3 Compare February 20, 2023 15:49
wzshiming
wzshiming reviewed Feb 20, 2023
View reviewed changes
test/release/build.test.sh Outdated Show resolved Hide resolved
images/kwok/build.sh Show resolved Hide resolved
@zwpaper zwpaper force-pushed the build-nerdctl-without-docker branch from 94622b3 to dd27508 Compare February 21, 2023 16:09
@zwpaper
test: test build on docker and nerdctl
899e6b9 
Signed-off-by: Wei Zhang <kweizh@gmail.com>
@zwpaper zwpaper force-pushed the build-nerdctl-without-docker branch from dd27508 to 899e6b9 Compare February 21, 2023 16:37
@wzshiming
Copy link
Member

/approve
/lgtm

Thank you for your contribution

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Feb 22, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wzshiming, zwpaper

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 1423efb into kubernetes-sigs:main Feb 22, 2023
@zwpaper zwpaper mentioned this pull request Mar 3, 2023
Closed
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wzshiming wzshiming wzshiming approved these changes

@yibozhuang yibozhuang Awaiting requested review from yibozhuang

Assignees

@wzshiming wzshiming

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@zwpaper @k8s-ci-robot @wzshiming