Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable new log dumping mechanism for all scalability periodic jobs #19710

Merged
merged 3 commits into from
Oct 28, 2020
Merged

Enable new log dumping mechanism for all scalability periodic jobs #19710

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2bc42f6
Rename scalability periodics preset for master branch jobs
tosi3k Oct 27, 2020
3d526fa
Add scalability periodics preset for all branches
tosi3k Oct 27, 2020
d92ceab
Enable new log dumping mechanism for all scalability periodic jobs
tosi3k Oct 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions config/jobs/kubernetes/sig-release/release-branch-jobs/1.16.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,7 @@ periodics:
cron: 0 8-20/12 * * *
labels:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-periodics: "true"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly as in other PRs - can we split release branches from the master?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is already split (at the PR level), please check individual commits here.

preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-e2e-gci-gce-scalability-stable3
Expand All @@ -214,6 +215,14 @@ periodics:
- --gcp-project-type=scalability-project
- --gcp-zone=us-east1-b
- --provider=gce
# Logexporter creates a daemonset per compute zone to extract logs from
# nodes in each zone separately. This mechanism relies on
# topology.kubernetes.io/zone node label that was added in K8s 1.17.
# In older releases, failure-domain.beta.kubernetes.io/zone label is used
# instead. As the former label is used by the logexporter by default, we
# override it by setting ZONE_NODE_SELECTOR_LABEL env var to the latter one.
# More context: https://github.com/kubernetes/kubernetes/pull/94056
- --env=ZONE_NODE_SELECTOR_LABEL=failure-domain.beta.kubernetes.io/zone
- --test=false
- --test-cmd=$GOPATH/src/k8s.io/perf-tests/run-e2e.sh
- --test-cmd-args=cluster-loader2
Expand Down Expand Up @@ -252,6 +261,7 @@ periodics:
labels:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-kubemark-500-gce-stable3
Expand All @@ -275,6 +285,14 @@ periodics:
- --kubemark
- --kubemark-nodes=500
- --provider=gce
# Logexporter creates a daemonset per compute zone to extract logs from
# nodes in each zone separately. This mechanism relies on
# topology.kubernetes.io/zone node label that was added in K8s 1.17.
# In older releases, failure-domain.beta.kubernetes.io/zone label is used
# instead. As the former label is used by the logexporter by default, we
# override it by setting ZONE_NODE_SELECTOR_LABEL env var to the latter one.
# More context: https://github.com/kubernetes/kubernetes/pull/94056
- --env=ZONE_NODE_SELECTOR_LABEL=failure-domain.beta.kubernetes.io/zone
- --test=false
- --test_args=--ginkgo.focus=xxxx
- --test-cmd=$GOPATH/src/k8s.io/perf-tests/run-e2e.sh
Expand Down
2 changes: 2 additions & 0 deletions config/jobs/kubernetes/sig-release/release-branch-jobs/1.17.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ periodics:
cron: 0 4-16/12 * * *
labels:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-periodics: "true"
preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-e2e-gci-gce-scalability-stable2
Expand Down Expand Up @@ -306,6 +307,7 @@ periodics:
labels:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-kubemark-500-gce-stable2
Expand Down
2 changes: 2 additions & 0 deletions config/jobs/kubernetes/sig-release/release-branch-jobs/1.18.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ periodics:
labels:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-kubemark-500-gce-stable1
Expand Down Expand Up @@ -303,6 +304,7 @@ periodics:
cron: 0 0/12 * * *
labels:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-periodics: "true"
preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-e2e-gci-gce-scalability-stable1
Expand Down
2 changes: 2 additions & 0 deletions config/jobs/kubernetes/sig-release/release-branch-jobs/1.19.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ periodics:
labels:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-kubemark-500-gce-beta
Expand Down Expand Up @@ -258,6 +259,7 @@ periodics:
labels:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-containerd: "true"
preset-e2e-scalability-periodics: "true"
preset-k8s-ssh: "true"
preset-service-account: "true"
name: ci-kubernetes-e2e-gci-gce-scalability-beta
Expand Down
6 changes: 6 additions & 0 deletions config/jobs/kubernetes/sig-scalability/sig-scalability-experimental-periodic-jobs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ periodics:
preset-k8s-ssh: "true"
preset-e2e-scalability-node: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-experiments
testgrid-tab-name: storage
Expand Down Expand Up @@ -49,6 +50,7 @@ periodics:
preset-k8s-ssh: "true"
preset-e2e-scalability-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-experiments
testgrid-tab-name: calico
Expand Down Expand Up @@ -107,6 +109,7 @@ periodics:
preset-k8s-ssh: "true"
preset-e2e-scalability-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-experiments
testgrid-tab-name: experimental-load
Expand Down Expand Up @@ -161,6 +164,7 @@ periodics:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-containerd: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-experiments
testgrid-tab-name: experimental-load-containerd
Expand Down Expand Up @@ -212,6 +216,7 @@ periodics:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-containerd: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-experiments
testgrid-tab-name: gce-cos-master-scalability-100-containerd-correctness
Expand Down Expand Up @@ -252,6 +257,7 @@ periodics:
preset-k8s-ssh: "true"
preset-e2e-scalability-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-experiments
testgrid-tab-name: gce-cos-master-scalability-100-nodekiller
Expand Down
1 change: 1 addition & 0 deletions config/jobs/kubernetes/sig-scalability/sig-scalability-golang.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ periodics:
preset-e2e-kubemark-common: "true"
preset-e2e-kubemark-gce-scale: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-alert-email: go-kubernetes-scalability-tickets@googlegroups.com
testgrid-dashboards: sig-scalability-golang
Expand Down
11 changes: 11 additions & 0 deletions config/jobs/kubernetes/sig-scalability/sig-scalability-periodic-jobs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ periodics:
preset-k8s-ssh: "true"
preset-e2e-scalability-node: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-node
testgrid-tab-name: node-throughput
Expand Down Expand Up @@ -58,6 +59,7 @@ periodics:
preset-e2e-scalability-node: "true"
preset-e2e-scalability-containerd: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-node
testgrid-tab-name: node-containerd-throughput
Expand Down Expand Up @@ -110,6 +112,7 @@ periodics:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-kubemark
testgrid-tab-name: kubemark-100
Expand Down Expand Up @@ -177,6 +180,7 @@ periodics:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-kubemark
testgrid-tab-name: kubemark-100-scheduler
Expand Down Expand Up @@ -242,6 +246,7 @@ periodics:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
fork-per-release: "true"
fork-per-release-cron: 0 3 * * *, 0 9 * * *, 0 15 * * *, 0 21 * * *
Expand Down Expand Up @@ -312,6 +317,7 @@ periodics:
preset-e2e-kubemark-common: "true"
preset-e2e-kubemark-gce-scale: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-kubemark
testgrid-tab-name: kubemark-5000
Expand Down Expand Up @@ -384,6 +390,7 @@ periodics:
preset-e2e-kubemark-common: "true"
preset-e2e-kubemark-gce-scale: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-kubemark
testgrid-tab-name: kubemark-5000-scheduler
Expand Down Expand Up @@ -445,6 +452,7 @@ periodics:
preset-dind-enabled: "true"
preset-e2e-kubemark-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-kubemark
testgrid-tab-name: kubemark-100-high-density
Expand Down Expand Up @@ -502,6 +510,7 @@ periodics:
preset-service-account: "true"
preset-k8s-ssh: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-perf-tests
testgrid-tab-name: kubemark-100-benchmark
Expand Down Expand Up @@ -612,6 +621,7 @@ periodics:
preset-service-account: "true"
preset-k8s-ssh: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-benchmarks
testgrid-tab-name: kube-dns
Expand Down Expand Up @@ -655,6 +665,7 @@ periodics:
preset-service-account: "true"
preset-k8s-ssh: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-dashboards: sig-scalability-benchmarks
testgrid-tab-name: node-local-dns
Expand Down
4 changes: 4 additions & 0 deletions config/jobs/kubernetes/sig-scalability/sig-scalability-presets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -288,3 +288,7 @@ presets:
# instead of relying on the deprecated one from k/k repository.
- name: USE_KUBEKINS_LOG_DUMPING
value: "yes"

- labels:
preset-e2e-scalability-periodics-master: "true"
env:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm - it seems that it has already been enabled before, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean for master branch...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that was this other PR you were mentioning

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#19663

3 changes: 3 additions & 0 deletions config/jobs/kubernetes/sig-scalability/sig-scalability-release-blocking-jobs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ periodics:
preset-k8s-ssh: "true"
preset-e2e-scalability-common: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-alert-email: kubernetes-sig-scale@googlegroups.com, kubernetes-scalability-tickets@google.com, release-team@kubernetes.io
testgrid-dashboards: sig-release-master-informing, sig-scalability-gce, google-gce
Expand Down Expand Up @@ -58,6 +59,7 @@ periodics:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-containerd: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
testgrid-alert-email: kubernetes-sig-scale@googlegroups.com, kubernetes-scalability-tickets@google.com, release-team@kubernetes.io
testgrid-dashboards: sig-release-master-informing, sig-scalability-gce, google-gce
Expand Down Expand Up @@ -124,6 +126,7 @@ periodics:
preset-e2e-scalability-common: "true"
preset-e2e-scalability-containerd: "true"
preset-e2e-scalability-periodics: "true"
preset-e2e-scalability-periodics-master: "true"
annotations:
fork-per-release: "true"
fork-per-release-cron: 0 */6 * * *, 0 0/12 * * *, 0 4-16/12 * * *, 0 8-20/12 * * *
Expand Down