Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add --extract-{ci,release}-bucket flags to kubetest #19620

Merged
merged 2 commits into from
Oct 20, 2020
Merged

Add --extract-{ci,release}-bucket flags to kubetest #19620

merged 2 commits into from
Oct 20, 2020

Conversation

spiffxp
Copy link
Member

@spiffxp spiffxp commented Oct 19, 2020

This is to support migrating jobs to community-owned buckets:

  • ci: kubernetes-release-dev -> k8s-release-dev
  • release: kubernetes-release -> k8s-release

The flags default to the google-owned kubernetes buckets, so jobs
will require config changes to migrate

kubetest ends up setting KUBERNETES_RELEASE_URL and calling out
to kubernetes/kubernetes/cluster/get-kube.sh, which will then
use the env var instead of any of its built-in defaults

ref:
kubernetes/k8s.io#846 (comment)
ref: #19484 (comment)

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 19, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 19, 2020
@k8s-ci-robot k8s-ci-robot added area/kubetest sig/testing Categorizes an issue or PR as relevant to SIG Testing. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 19, 2020
@spiffxp
Add --extract-{ci,release}-bucket flags to kubetest
d943f81 
This is to support migrating jobs to community-owned buckets:

- ci: kubernetes-release-dev -> k8s-release-dev
- release: kubernetes-release -> k8s-release

The flags default to the google-owned kubernetes buckets, so jobs
will require config changes to migrate

kubetest ends up setting KUBERNETES_RELEASE_URL and calling out
to kubernetes/kubernetes/cluster/get-kube.sh, which will then
use the env var instead of any of its built-in defaults
@spiffxp spiffxp force-pushed the add-extract-bucket-kubetest-options branch from 1ebe169 to d943f81 Compare October 19, 2020 19:47
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 19, 2020
amwat
amwat reviewed Oct 19, 2020
View reviewed changes
Copy link
Contributor

@amwat amwat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/hold

kubetest/main.go Outdated
@@ -135,6 +137,8 @@ func defineFlags() *options {
flag.StringVar(&o.dump, "dump", "", "If set, dump bring-up and cluster logs to this location on test or cluster-up failure")
flag.StringVar(&o.dumpPreTestLogs, "dump-pre-test-logs", "", "If set, dump cluster logs to this location before running tests")
flag.Var(&o.extract, "extract", "Extract k8s binaries from the specified release location")
flag.StringVar(&o.extractCIBucket, "extract-ci-bucket", "kubernetes-release-dev", "Extract k8s CI binaries from the specified GCS bucket")
flag.StringVar(&o.extractReleaseBucket, "extract-release-bucket", "kubernetes-release", "Extract k8s CI binaries from the specified GCS bucket")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
flag.StringVar(&o.extractReleaseBucket, "extract-release-bucket", "kubernetes-release", "Extract k8s CI binaries from the specified GCS bucket")
flag.StringVar(&o.extractReleaseBucket, "extract-release-bucket", "kubernetes-release", "Extract k8s release binaries from the specified GCS bucket")

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks more like we need a way to override the two default hardcoded values, and not as much we need a way to customize both values in a specific run? Do we want to consider using experimental envs instead?

A more involved change would modify the corresponding strategies, but given the current maintenance state this minimal change LGTM.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks more like we need a way to override the two default hardcoded values, and not as much we need a way to customize both values in a specific run?

Migrating kubernetes-release is going to be more involved, so I was anticipating upgrade jobs for an existing release to the in-development release pulling from kubernetes-release and k8s-release-dev in the interim.

Do we want to consider using experimental envs instead?

I'd rather avoid envs and would prefer flags for less magic. Though admittedly kubernetes/kubernetes scripts by themselves will need envs eventually.

A more involved change would modify the corresponding strategies, but given the current maintenance state this minimal change LGTM.

Agreed. The current strategies parse themselves from the string passed to extract. I considered something like k8s-release-dev:ci/latest, but as you said, that seemed like it more be more invasive.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 19, 2020
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 19, 2020
@spiffxp @amwat
Update kubetest/main.go
dd3df3b 
Co-authored-by: Amit Watve <amwat@google.com>
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 20, 2020
@amwat
Copy link
Contributor

amwat commented Oct 20, 2020

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 20, 2020
@amwat
Copy link
Contributor

amwat commented Oct 20, 2020

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 20, 2020
@k8s-ci-robot k8s-ci-robot merged commit 9838966 into kubernetes:master Oct 20, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.20 milestone Oct 20, 2020
This was referenced Oct 20, 2020
Merged
Closed
@spiffxp spiffxp deleted the add-extract-bucket-kubetest-options branch October 20, 2020 20:25
@spiffxp spiffxp mentioned this pull request Oct 20, 2020
Merged
@spiffxp spiffxp mentioned this pull request Nov 20, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amwat amwat amwat left review comments

@shyamjvs shyamjvs Awaiting requested review from shyamjvs

@wojtek-t wojtek-t Awaiting requested review from wojtek-t

Assignees

@amwat amwat

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubetest cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
v1.20
Development

Successfully merging this pull request may close these issues.
3 participants
@spiffxp @k8s-ci-robot @amwat