Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kubernete-CSI: set snapshot CRD version #19939

Merged
merged 2 commits into from
Nov 16, 2020
Merged

Kubernete-CSI: set snapshot CRD version #19939

merged 2 commits into from
Nov 16, 2020

Conversation

pohly
Copy link
Contributor

@pohly pohly commented Nov 13, 2020

Some jobs need the "master" version of the snapshotter CRD and
controller. For others it makes sense to select the version that gets
tested in the Prow job instead of having to bump the prow.sh script in
all affected repos.

The caveat is that all future versions of external-snapshotter must
install through exactly the YAML files that are currently used by
prow.sh:
https://github.com/kubernetes-csi/csi-release-tools/blob/5d874cce4e649dfd254d01b9b44179ffa72aee75/prow.sh#L702-L722

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 13, 2020
@k8s-ci-robot k8s-ci-robot added area/config Issues or PRs related to code in /config area/jobs sig/testing Categorizes an issue or PR as relevant to SIG Testing. labels Nov 13, 2020
@pohly
Copy link
Contributor Author

pohly commented Nov 13, 2020

/assign @msau42
/cc @xing-yang

Here's a different approach for bumping the CRD version.

If the install URLs are stable across releases, then this should be viable. I'll let you guys decide whether that is acceptable.

pohly added 2 commits November 13, 2020 13:00
@pohly
kubernetes-csi: move manual csi-driver-nfs job
1145b61 
Commit 0ad87ab added a manually
defined job to a file that gets overrwritten by gen-jobs.sh.
The manual job has to be in a separate file.
@pohly
kubernetes-csi: set snapshotter version
440de3d 
Some jobs need the "master" version of the snapshotter CRD and
controller. For others it makes sense to select the version that gets
tested in the Prow job instead of having to bump the prow.sh script in
all affected repos.

The caveat is that all future versions of external-snapshotter must
install through exactly the YAML files that are currently used by
prow.sh:
https://github.com/kubernetes-csi/csi-release-tools/blob/5d874cce4e649dfd254d01b9b44179ffa72aee75/prow.sh#L702-L722
@pohly pohly force-pushed the snapshoter-canary branch from cec746d to 440de3d Compare November 13, 2020 12:00
@pohly pohly mentioned this pull request Nov 13, 2020
Merged
xing-yang
xing-yang reviewed Nov 13, 2020
View reviewed changes
config/jobs/kubernetes-csi/csi-driver-nfs/csi-driver-nfs-config.yaml
# docker-in-docker needs privileged mode
securityContext:
privileged: true
resources:
requests:
cpu: 2000m
- name: pull-csi-driver-nfs-e2e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change required for the snapshot CRD version changes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to fix a mistake from someone's prior PR, see the first commit in this PR: 1145b61

config/jobs/kubernetes-csi/external-health-monitor/external-health-monitor-config.yaml
@@ -23,6 +23,9 @@ presubmits:
- runner.sh
args:
- ./.prow.sh
env:
- name: CSI_SNAPSHOTTER_VERSION
value: "v3.0.2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be "master"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job runs with the Kubernetes and sidecar version that is specified inside the prow.sh job, which shouldn't depend on an unreleased snapshotter. So using the fixed version seems fine here.

One could also argue that this particular job shouldn't override CSI_SNAPSHOTTER_VERSION.

@xing-yang
Copy link
Contributor

I'm fine with this approach.
@msau42 what do you think?

@pohly
Copy link
Contributor Author

pohly commented Nov 13, 2020

/retest

msau42
msau42 reviewed Nov 13, 2020
View reviewed changes
config/jobs/kubernetes-csi/external-snapshotter/external-snapshotter-config.yaml
@@ -36,6 +36,8 @@ presubmits:
value: "1.17"
- name: CSI_PROW_DRIVER_VERSION
value: "v1.4.0"
- name: CSI_SNAPSHOTTER_VERSION
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean that changes to the snapshot-controller in external-snapshotter prs are untested?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is tested: https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_external-snapshotter/425/pull-kubernetes-csi-external-snapshotter-1-17-on-kubernetes-1-17/1326879789335711744/artifacts/default/csi-hostpath-snapshotter-0/csi-snapshotter.log

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-csi_external-snapshotter/425/pull-kubernetes-csi-external-snapshotter-1-17-on-kubernetes-1-17/1326879789335711744/artifacts/default/snapshot-controller-0/snapshot-controller.log

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CSI_SNAPSHOTTER_VERSION is set in prow.sh currently.
https://github.com/kubernetes-csi/csi-release-tools/blob/master/prow.sh#L340

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is that it looks like prow.sh installs a version of the snapshot controller that has already merged and been released? https://github.com/kubernetes-csi/csi-release-tools/blob/master/prow.sh#L722

So if you make changes to snapshot-controller in a PR, then those changes won't get tested as part of the pull job.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like that's the case. We probably need to replace the snapshot-controller image with the one built from the PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For images used by the csi-driver-host-path deploy script, overriding the images that were built as part of the repo under test is defined here:
https://github.com/kubernetes-csi/csi-release-tools/blob/5d874cce4e649dfd254d01b9b44179ffa72aee75/prow.sh#L1047-L1079

Something similar is needed for the snapshot controller and prow.sh itself.

But this is unrelated to this PR, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so. That will be a separate PR to fix prow.sh.

@msau42
Copy link
Member

msau42 commented Nov 16, 2020

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 16, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: msau42, pohly

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 16, 2020
@k8s-ci-robot k8s-ci-robot merged commit eea03c9 into kubernetes:master Nov 16, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.20 milestone Nov 16, 2020
@k8s-ci-robot
Copy link
Contributor

@pohly: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

  • key csi-driver-flex-config.yaml using file config/jobs/kubernetes-csi/csi-driver-flex/csi-driver-flex-config.yaml
  • key csi-driver-host-path-config.yaml using file config/jobs/kubernetes-csi/csi-driver-host-path/csi-driver-host-path-config.yaml
  • key csi-driver-iscsi-config.yaml using file config/jobs/kubernetes-csi/csi-driver-iscsi/csi-driver-iscsi-config.yaml
  • key csi-driver-nfs-config.yaml using file config/jobs/kubernetes-csi/csi-driver-nfs/csi-driver-nfs-config.yaml
  • key csi-driver-nfs-unmanaged.yaml using file config/jobs/kubernetes-csi/csi-driver-nfs/csi-driver-nfs-unmanaged.yaml
  • key csi-lib-utils-config.yaml using file config/jobs/kubernetes-csi/csi-lib-utils/csi-lib-utils-config.yaml
  • key csi-proxy-config.yaml using file config/jobs/kubernetes-csi/csi-proxy/csi-proxy-config.yaml
  • key csi-release-tools-config.yaml using file config/jobs/kubernetes-csi/csi-release-tools/csi-release-tools-config.yaml
  • key csi-test-config.yaml using file config/jobs/kubernetes-csi/csi-test/csi-test-config.yaml
  • key external-attacher-config.yaml using file config/jobs/kubernetes-csi/external-attacher/external-attacher-config.yaml
  • key external-health-monitor-config.yaml using file config/jobs/kubernetes-csi/external-health-monitor/external-health-monitor-config.yaml
  • key external-provisioner-config.yaml using file config/jobs/kubernetes-csi/external-provisioner/external-provisioner-config.yaml
  • key external-resizer-config.yaml using file config/jobs/kubernetes-csi/external-resizer/external-resizer-config.yaml
  • key external-snapshotter-config.yaml using file config/jobs/kubernetes-csi/external-snapshotter/external-snapshotter-config.yaml
  • key livenessprobe-config.yaml using file config/jobs/kubernetes-csi/livenessprobe/livenessprobe-config.yaml
  • key node-driver-registrar-config.yaml using file config/jobs/kubernetes-csi/node-driver-registrar/node-driver-registrar-config.yaml

In response to this:

Some jobs need the "master" version of the snapshotter CRD and
controller. For others it makes sense to select the version that gets
tested in the Prow job instead of having to bump the prow.sh script in
all affected repos.

The caveat is that all future versions of external-snapshotter must
install through exactly the YAML files that are currently used by
prow.sh:
https://github.com/kubernetes-csi/csi-release-tools/blob/5d874cce4e649dfd254d01b9b44179ffa72aee75/prow.sh#L702-L722

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@msau42 msau42 msau42 left review comments

@xing-yang xing-yang xing-yang left review comments

@andyzhangx andyzhangx Awaiting requested review from andyzhangx

Assignees

@msau42 msau42

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/config Issues or PRs related to code in /config area/jobs cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.20
Development

Successfully merging this pull request may close these issues.
4 participants
@pohly @xing-yang @msau42 @k8s-ci-robot