Move all NFD Prow jobs to Community cluster #32887

ArangoGutierrez · 2024-07-02T14:40:39Z

As seen at https://groups.google.com/a/kubernetes.io/g/dev/c/p6PAML90ZOU

ArangoGutierrez · 2024-07-02T14:40:52Z

/cc @BenTheElder @marquiz

ArangoGutierrez · 2024-07-02T14:41:41Z

Fixes kubernetes-sigs/node-feature-discovery#1747

ArangoGutierrez · 2024-07-02T15:01:44Z

/test pull-test-infra-verify-lint

BenTheElder · 2024-07-02T16:33:00Z

config/jobs/kubernetes-sigs/node-feature-discovery/node-feature-discovery-postsubmits.yaml

  - name: postsubmit-node-feature-discovery-e2e-test
+    cluster: eks-prow-build-cluster


if you're doing GCP e2e tests you should use the GCP cluster (k8s-infra-prow-build)

(other jobs like unit test / verify, AWS e2e can and should use the EKS cluster to spread the load between the cloud credits)

BenTheElder · 2024-07-02T16:33:43Z

failing linters

BenTheElder · 2024-07-02T17:12:50Z

config/jobs/kubernetes-sigs/node-feature-discovery/node-feature-discovery-postsubmits.yaml

@@ -1,6 +1,7 @@
 postsubmits:
  kubernetes-sigs/node-feature-discovery:
  - name: postsubmit-node-feature-discovery-verify-master
+    cluster: k8s-infra-prow-build


this one should probably be on the EKS prow, and it must have been moved there previously as it's using public.ecr.aws/docker/library/golang:1.22

it's not doing e2e is it?

https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/scripts/test-infra/verify.sh
so yeah, not e2e

should I move it to EKS?

BenTheElder · 2024-07-02T17:13:29Z

/approve
/hold

BenTheElder · 2024-07-02T17:24:04Z

config/jobs/kubernetes-sigs/node-feature-discovery/node-feature-discovery-postsubmits.yaml

@@ -59,3 +68,10 @@ postsubmits:
          value: '{"apiVersion":"v1","clusters":[{"cluster":{"certificate-authority-data":"LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMvakNDQWVhZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeU1EWXpNREEzTURjek1sb1hEVE15TURZeU56QTNNRGN6TWxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS1krClQzaWREc2E4L1Z4RkRxREEzRWFzSEk1VTFYcWZmZFhVb25qTXFxeDJpM21WcGNLa1dhTGdwcVZoZ3YzdlJPbi8KVlltcVlYRUNaOTBEOHhKMGFIbTMrMElmekJJRkJSdTY5UWRmUSs2ZXVlUldZVzh3YnVVYjdnZzdyNXQreHZYNApQSFNvMVFSQ001dWw5dFJrd2w3Rm5wcVc0dW5mYnUxaERMdGNzeTMrZkM2eHBDVi9RUVpuQ0ZsbjhkUTQ0cUZICkZNd3NLd1AyZkxpU0FveHRNT3c4TGlpSkk5QWhEb3BIUXNOOEVJQkpHSGN0dDYyb1c2aklKUmhRelFpTnIyaFgKalZ6OFBaVGN0YVZNR1pNbzJFVllaMlIwYTZuSmcyOGVtRUJNL1MyemtBYmoxR3kwUitHanVwRmQ3RGRaUkRFaApHaVpXZXFnNUpyU1lLTWtJejBVQ0F3RUFBYU5aTUZjd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZPUE5QODJJcFQ2OW1GQkg5QWwwTVNEbStNVDZNQlVHQTFVZEVRUU8KTUF5Q0NtdDFZbVZ5Ym1WMFpYTXdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBQlhhUHIvenpyMXNjZFgyMFB3Rwo0bWxZQVhpRkdCUkYrQndyWVNTOEVRR2NMVXo5eHhUNEZ1aUQ5MmNBcDQvWTlPWG5DTEFvYkNBN0x2QTN0elpMCm9lUE01U1Jyb1Q3MythZzJ0RHBuWDEyeFB6aEpGTzArVkhGeENSMkhTOHArY1F0eC94Ym1MRTZMbzNJS3dMSisKVEVTT1F3cUNHSmxNNHNiVk9jU0ZGeFRzUURid1hpblVyN1ZqSm03UW9LVHEvcGIwWjU3Q3NlWXYzd1VTYXdWVAo3aXVVdHBMWmhrMDdxZ09XYzFNTzFoNlUyU3BNbkY1akVnSXNmSlByRUJic3RkME55STZFN0FBRzNOSGZOczJlCitHNEhDS0tRdkc0emJkdmFwWjJOSVRaM0NJaml4b0lGOXNDb3pDYWVWV0lCT0xib2lhcVp3M3hyOTZtdnlNNWIKZkNvPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","server":"https://1AF488C80378BE347035F6704C6F0C8F.gr7.eu-west-1.eks.amazonaws.com"},"name":"aws"}],"contexts":[{"context":{"cluster":"aws","user":"prow"},"name":"aws"}],"current-context":"aws","kind":"Config","preferences":{},"users":[{"name":"prow","user":{"exec":{"apiVersion":"client.authentication.k8s.io/v1","args":["token","-i","nfd-e2e"],"command":"aws-iam-authenticator","interactiveMode":"Never"}}}]}'
        - name: E2E_TEST_CONFIG_DATA
          value: '{"kubelet":{"configPath":"/etc/kubernetes/kubelet/kubelet-config.json"},"defaultFeatures":{"labelWhitelist":["feature.node.kubernetes.io/cpu-cpuid.ADX","feature.node.kubernetes.io/cpu-cpuid.AESNI","feature.node.kubernetes.io/cpu-cpuid.AMXBF16D","feature.node.kubernetes.io/cpu-cpuid.AMXINT8","feature.node.kubernetes.io/cpu-cpuid.AMXTILE","feature.node.kubernetes.io/cpu-cpuid.AVX","feature.node.kubernetes.io/cpu-cpuid.AVX2","feature.node.kubernetes.io/cpu-cpuid.AVX512BF16","feature.node.kubernetes.io/cpu-cpuid.AVX512BITALG","feature.node.kubernetes.io/cpu-cpuid.AVX512BW","feature.node.kubernetes.io/cpu-cpuid.AVX512CD","feature.node.kubernetes.io/cpu-cpuid.AVX512DQ","feature.node.kubernetes.io/cpu-cpuid.AVX512ER","feature.node.kubernetes.io/cpu-cpuid.AVX512F","feature.node.kubernetes.io/cpu-cpuid.AVX512FP16","feature.node.kubernetes.io/cpu-cpuid.AVX512IFMA","feature.node.kubernetes.io/cpu-cpuid.AVX512PF","feature.node.kubernetes.io/cpu-cpuid.AVX512VBMI","feature.node.kubernetes.io/cpu-cpuid.AVX512VBMI2","feature.node.kubernetes.io/cpu-cpuid.AVX512VL","feature.node.kubernetes.io/cpu-cpuid.AVX512VNNI","feature.node.kubernetes.io/cpu-cpuid.AVX512VP2INTERSECT","feature.node.kubernetes.io/cpu-cpuid.AVX512VPOPCNTDQ","feature.node.kubernetes.io/cpu-cpuid.ENQCMD","feature.node.kubernetes.io/cpu-cpuid.FMA3","feature.node.kubernetes.io/cpu-cpuid.GFNI","feature.node.kubernetes.io/cpu-cpuid.HLE","feature.node.kubernetes.io/cpu-cpuid.HYPERVISOR","feature.node.kubernetes.io/cpu-cpuid.IBPB","feature.node.kubernetes.io/cpu-cpuid.MPX","feature.node.kubernetes.io/cpu-cpuid.PCONFIG","feature.node.kubernetes.io/cpu-cpuid.RTM","feature.node.kubernetes.io/cpu-cpuid.SHA","feature.node.kubernetes.io/cpu-cpuid.STIBP","feature.node.kubernetes.io/cpu-cpuid.TME","feature.node.kubernetes.io/cpu-cpuid.VAES","feature.node.kubernetes.io/cpu-cpuid.VMX","feature.node.kubernetes.io/cpu-cpuid.VPCLMULQDQ","feature.node.kubernetes.io/cpu-cstate.enabled","feature.node.kubernetes.io/cpu-hardware_multithreading","feature.node.kubernetes.io/cpu-model.family","feature.node.kubernetes.io/cpu-model.id","feature.node.kubernetes.io/cpu-model.vendor_id","feature.node.kubernetes.io/cpu-power.sst_bf.enabled","feature.node.kubernetes.io/cpu-pstate.status","feature.node.kubernetes.io/cpu-pstate.scaling_governor","feature.node.kubernetes.io/cpu-pstate.turbo","feature.node.kubernetes.io/cpu-rdt.RDTCMT","feature.node.kubernetes.io/cpu-rdt.RDTL3CA","feature.node.kubernetes.io/cpu-rdt.RDTMBA","feature.node.kubernetes.io/cpu-rdt.RDTMBM","feature.node.kubernetes.io/cpu-rdt.RDTMON","feature.node.kubernetes.io/iommu-enabled","feature.node.kubernetes.io/kernel-config.NO_HZ","feature.node.kubernetes.io/kernel-config.NO_HZ_FULL","feature.node.kubernetes.io/kernel-config.NO_HZ_IDLE","feature.node.kubernetes.io/kernel-config.PREEMPT","feature.node.kubernetes.io/kernel-selinux.enabled","feature.node.kubernetes.io/kernel-version.full","feature.node.kubernetes.io/kernel-version.major","feature.node.kubernetes.io/kernel-version.minor","feature.node.kubernetes.io/kernel-version.revision","feature.node.kubernetes.io/memory-numa","feature.node.kubernetes.io/memory-nv.dax","feature.node.kubernetes.io/memory-nv.present","feature.node.kubernetes.io/network-sriov.capable","feature.node.kubernetes.io/network-sriov.configured","feature.node.kubernetes.io/pci-0300_1a03.present","feature.node.kubernetes.io/storage-nonrotationaldisk","feature.node.kubernetes.io/system-os_release.ID","feature.node.kubernetes.io/system-os_release.VERSION_ID","feature.node.kubernetes.io/system-os_release.VERSION_ID.major","feature.node.kubernetes.io/system-os_release.VERSION_ID.minor"],"nodes":[{"name":"default","nodeNameRegexp":".","expectedLabelValues":{"feature.node.kubernetes.io/cpu-cpuid.ADX":"true","feature.node.kubernetes.io/cpu-cpuid.AESNI":"true","feature.node.kubernetes.io/cpu-cpuid.AVX":"true","feature.node.kubernetes.io/cpu-cpuid.AVX2":"true","feature.node.kubernetes.io/cpu-cpuid.AVX512BW":"true","feature.node.kubernetes.io/cpu-cpuid.AVX512CD":"true","feature.node.kubernetes.io/cpu-cpuid.AVX512DQ":"true","feature.node.kubernetes.io/cpu-cpuid.AVX512F":"true","feature.node.kubernetes.io/cpu-cpuid.AVX512VL":"true","feature.node.kubernetes.io/cpu-cpuid.FMA3":"true","feature.node.kubernetes.io/cpu-cpuid.HYPERVISOR":"true","feature.node.kubernetes.io/cpu-cpuid.MPX":"true","feature.node.kubernetes.io/cpu-hardware_multithreading":"true","feature.node.kubernetes.io/cpu-model.family":"6","feature.node.kubernetes.io/cpu-model.id":"85","feature.node.kubernetes.io/cpu-model.vendor_id":"Intel","feature.node.kubernetes.io/kernel-config.NO_HZ":"true","feature.node.kubernetes.io/kernel-config.NO_HZ_IDLE":"true","feature.node.kubernetes.io/kernel-version.major":"5","feature.node.kubernetes.io/pci-0300_1d0f.present":"true","feature.node.kubernetes.io/storage-nonrotationaldisk":"true","feature.node.kubernetes.io/system-os_release.ID":"amzn","feature.node.kubernetes.io/system-os_release.VERSION_ID.major":"2"},"expectedLabelKeys":["feature.node.kubernetes.io/kernel-version.full","feature.node.kubernetes.io/kernel-version.major","feature.node.kubernetes.io/kernel-version.minor","feature.node.kubernetes.io/kernel-version.revision","feature.node.kubernetes.io/system-os_release.VERSION_ID"],"expectedAnnotationKeys":["nfd.node.kubernetes.io/feature-labels"]}]}}'
+        resources:


TODO: evaluate the necessary amount for this job (happy to defer that and get it migrated first)

for our e2e we have tried 2 cpu's but it runs out of time. 4C/8M has been the min set , but happy to evaluate if we can trim down a bit

ArangoGutierrez · 2024-07-02T18:05:37Z

/test pull-test-infra-verify-lint

ArangoGutierrez · 2024-07-02T18:06:15Z

failing linters

the lints that trigger the failure are not on NFD files, how can we pass by this, should I rebase on master or something?

Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

BenTheElder

/lgtm
/approve

k8s-ci-robot · 2024-07-02T18:10:32Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ArangoGutierrez, BenTheElder

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~config/jobs/kubernetes-sigs/node-feature-discovery/OWNERS~~ [ArangoGutierrez,BenTheElder]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ArangoGutierrez · 2024-07-02T18:13:43Z

/test pull-test-infra-unit-test

BenTheElder · 2024-07-02T18:20:22Z

/hold cancel
/retest

BenTheElder · 2024-07-02T18:37:39Z

config/jobs/kubernetes-sigs/node-feature-discovery/node-feature-discovery-postsubmits.yaml
  31:24     error    trailing spaces  (trailing-spaces)
  77:26     error    no new line character at the end of file  (new-line-at-end-of-file)
  77:24     error    trailing spaces  (trailing-spaces)

https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/test-infra/32887/pull-test-infra-verify-lint/1808204092611235840

BenTheElder · 2024-07-02T18:38:07Z

the lints that trigger the failure are not on NFD files, how can we pass by this, should I rebase on master or something?

the other lints are warnings, which are not the failure, NFD is failing (see above, you can click to expand the log lines for more detail)

Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

ArangoGutierrez · 2024-07-02T18:39:46Z

the lints that trigger the failure are not on NFD files, how can we pass by this, should I rebase on master or something?

the other lints are warnings, which are not the failure, NFD is failing (see above, you can click to expand the log lines for more detail)

Fixed

BenTheElder

/lgtm

k8s-ci-robot · 2024-07-02T18:50:31Z

@ArangoGutierrez: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

key node-feature-discovery-postsubmits.yaml using file config/jobs/kubernetes-sigs/node-feature-discovery/node-feature-discovery-postsubmits.yaml
key node-feature-discovery-presubmits-master.yaml using file config/jobs/kubernetes-sigs/node-feature-discovery/node-feature-discovery-presubmits-master.yaml
key node-feature-discovery-presubmits-release-0-16.yaml using file config/jobs/kubernetes-sigs/node-feature-discovery/node-feature-discovery-presubmits-release-0-16.yaml

In response to this:

As seen at https://groups.google.com/a/kubernetes.io/g/dev/c/p6PAML90ZOU

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. area/config Issues or PRs related to code in /config area/jobs labels Jul 2, 2024

k8s-ci-robot added the sig/testing Categorizes an issue or PR as relevant to SIG Testing. label Jul 2, 2024

k8s-ci-robot requested review from BenTheElder and marquiz July 2, 2024 14:40

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 2, 2024

k8s-ci-robot requested a review from kad July 2, 2024 14:40

ArangoGutierrez mentioned this pull request Jul 2, 2024

Migrate to Community CI By August 1st kubernetes-sigs/node-feature-discovery#1747

Closed

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 2, 2024

BenTheElder reviewed Jul 2, 2024

View reviewed changes

ArangoGutierrez force-pushed the community_cluster_nfd branch from 8c782e4 to 41901b9 Compare July 2, 2024 17:09

BenTheElder reviewed Jul 2, 2024

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 2, 2024

ArangoGutierrez requested a review from BenTheElder July 2, 2024 17:17

BenTheElder reviewed Jul 2, 2024

View reviewed changes

ArangoGutierrez force-pushed the community_cluster_nfd branch from 41901b9 to 4812557 Compare July 2, 2024 17:53

Move all NFD Prow jobs to Community cluster

fe43151

Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

ArangoGutierrez force-pushed the community_cluster_nfd branch from 4812557 to 3b3cf7c Compare July 2, 2024 18:06

BenTheElder reviewed Jul 2, 2024

View reviewed changes

k8s-ci-robot assigned BenTheElder Jul 2, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 2, 2024

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 2, 2024

Add resource limits to NFD Postsubmit jobs

01b9983

Signed-off-by: Carlos Eduardo Arango Gutierrez <eduardoa@nvidia.com>

ArangoGutierrez force-pushed the community_cluster_nfd branch from 3b3cf7c to 01b9983 Compare July 2, 2024 18:38

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 2, 2024

BenTheElder reviewed Jul 2, 2024

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 2, 2024

k8s-ci-robot merged commit 27ef1ce into kubernetes:master Jul 2, 2024
7 checks passed

upodroid mentioned this pull request Jul 5, 2024

Create nfd codecov token secret kubernetes/k8s.io#6948

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move all NFD Prow jobs to Community cluster #32887

Move all NFD Prow jobs to Community cluster #32887

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

BenTheElder Jul 2, 2024

BenTheElder Jul 2, 2024

BenTheElder commented Jul 2, 2024

BenTheElder Jul 2, 2024

ArangoGutierrez Jul 2, 2024

ArangoGutierrez Jul 2, 2024

BenTheElder Jul 2, 2024

ArangoGutierrez Jul 2, 2024

BenTheElder commented Jul 2, 2024

BenTheElder Jul 2, 2024

ArangoGutierrez Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

BenTheElder left a comment

k8s-ci-robot commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

BenTheElder commented Jul 2, 2024

BenTheElder commented Jul 2, 2024

BenTheElder commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

BenTheElder left a comment

k8s-ci-robot commented Jul 2, 2024

		- name: postsubmit-node-feature-discovery-e2e-test
		cluster: eks-prow-build-cluster

Move all NFD Prow jobs to Community cluster #32887

Move all NFD Prow jobs to Community cluster #32887

Conversation

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BenTheElder commented Jul 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BenTheElder commented Jul 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ArangoGutierrez commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

BenTheElder left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

BenTheElder commented Jul 2, 2024

BenTheElder commented Jul 2, 2024

BenTheElder commented Jul 2, 2024

ArangoGutierrez commented Jul 2, 2024

BenTheElder left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 2, 2024