Windows upstream e2e #1119

jsturtevant · 2021-01-13T20:50:23Z

What type of PR is this?
/kind other

What this PR does / why we need it:
Add testing support for upstream windows tests similar to https://testgrid.k8s.io/sig-cluster-lifecycle-cluster-api-provider-azure#periodic-capz-conformance-k8s-master.

TODO:

Add ability to inject kube binaries similiar to https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/a3525d2d89821192dbaeef270f46eff09e4c4587/templates/test/prow-machine-pool-ci-version/patches/machine-pool-ci-version.yaml will follow up with separate pr. tracking in Add ability to inject kube binaries from kubernetes CI builds on Windows #1276
Create job in test-infra: Add windows upstream test job for capz kubernetes/test-infra#21544

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:
Some more work needs to be done to make this ready and it relies on:

Windows doesn't currently have Conformance tests but does have a set of tests that can verify Windows behavior. Defining Conformance tests for windows is a Work in progress and this set of tests can be updated later.

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

squashed commits
includes documentation
adds unit tests

Release note:

Windows e2e tests from kubernetes are run against capz workload cluster

k8s-ci-robot · 2021-01-13T20:50:25Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

k8s-ci-robot · 2021-01-13T20:50:25Z

@jsturtevant: The label(s) kind/other cannot be applied, because the repository doesn't have them

In response to this:

What type of PR is this?
/kind other

What this PR does / why we need it:
Add testing support for upstream windows tests similar to https://testgrid.k8s.io/sig-cluster-lifecycle-cluster-api-provider-azure#periodic-capz-conformance-k8s-master.

TODO: Add ability to inject kube binaries similiar to https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/a3525d2d89821192dbaeef270f46eff09e4c4587/templates/test/prow-machine-pool-ci-version/patches/machine-pool-ci-version.yaml

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:
Some more work needs to be done to make this ready and it relies on:

Windows doesn't currently have Conformance tests but does have a set of tests that can verify Windows behavior. Defining Conformance tests for windows is a Work in progress and this set of tests can be updated later.

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

squashed commits

includes documentation

adds unit tests

Release note:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jsturtevant · 2021-03-26T19:09:16Z

@CecileRobertMichon @nader-ziada This is ready for a review. I am running the suite now and will post results once it is complete. Sorry this took so long.

CecileRobertMichon · 2021-03-26T20:42:03Z

Added a comment in https://github.com/kubernetes/test-infra/pull/21544/files#r602575057, we should add a PR test equivalent to the periodic job and use it to validate this PR

jsturtevant · 2021-03-29T17:52:41Z

I got the test results back and had a few failures. Looking into today.

jsturtevant · 2021-04-09T22:58:59Z

update: there are a few tests that fail due to networking configuration. I am working with the networking team to get the correct settings.

jsturtevant · 2021-04-10T00:03:50Z

/test pull-cluster-api-provider-azure-upstream-v1alpha4-windows

devigned

Superficial nits

templates/test/ci/prow-machine-pool-windows/patches/windows-tmp-folder.yaml

templates/test/ci/prow-windows/patches/windows-tmp-folder.yaml

devigned · 2021-05-11T14:19:01Z

test/e2e/data/kubetest/repo-list.yaml

@@ -0,0 +1,3 @@
+gcAuthenticatedRegistry: e2eprivate
+gcEtcdRegistry: k8sprow.azurecr.io/kubernetes-e2e-test-images
+privateRegistry: e2eteam


new line nit

This was directly pulled during make from https://github.com/kubernetes-sigs/windows-testing/blob/master/images/image-repo-list-master. And it looks like that file doesn't have the file endings.

I checked it in here for an example file and so it was standalone.

jsturtevant · 2021-05-14T02:44:09Z

Will take a little longer to run but until kubernetes/test-infra#22180 merges but will be a signal nothing else changed and we can run it again for additional validation since this is a new test.

/test pull-cluster-api-provider-azure-upstream-v1alpha4-windows

nader-ziada · 2021-05-14T13:37:16Z

this one is good to go as well, @devigned any final thoughts since you had requested changes before

@jsturtevant 👏

jsturtevant · 2021-05-14T15:44:40Z

Running one last time with the updates in kubernetes/test-infra#22180

/test pull-cluster-api-provider-azure-upstream-v1alpha4-windows

jsturtevant · 2021-05-14T17:39:31Z

/test pull-cluster-api-provider-azure-upstream-v1alpha4-windows

jsturtevant · 2021-05-20T15:30:50Z

@devigned @nader-ziada @chewong any further thoughts on this?

CecileRobertMichon · 2021-05-20T16:25:16Z

templates/test/ci/cluster-template-prow-windows.yaml

@@ -298,6 +298,10 @@ spec:
          New-HnsNetwork -Type Overlay -AddressPrefix "192.168.255.0/30" -Gateway "192.168.255.1" -Name "External" -AdapterName "Ethernet 2" -SubnetPolicies @(@{Type = "VSID"; VSID = 9999; })
        path: C:/create-external-network.ps1
        permissions: "0744"
+      - content: |
+          New-Item -ItemType Directory -Force -Path C:\tmp\
+        path: C:/create-temp-folder.ps1


what is this temp folder useful for?

It is required for the end to end tests. It is assume to be already created on the machines for several of the tests. I swear I left a comment here but it looks like it might have been dropped during a rebase

add a comment

CecileRobertMichon

you might want to add to https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/master/docs/book/src/developers/development.md#conformance-testing

jsturtevant · 2021-05-20T17:37:45Z

you might want to add to https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/master/docs/book/src/developers/development.md#conformance-testing

Good call. Updated.

devigned

Outside of a naming question, lgtm

devigned · 2021-05-20T18:14:12Z

scripts/ci-conformance.sh

@@ -93,4 +94,8 @@ cleanup() {

 trap cleanup EXIT

-make test-conformance
+if [[ "${WINDOWS}" == "true" ]]; then
+  make test-windows-upstream


nit: what do you think about make test-conformance-windows?

I named it this way since there is no official "Conformance" for windows. We run a good set of conformance test but not all and additional Windows tests.

kubernetes/kubernetes#96639 for more info on where we are with Conformance for Windows

jsturtevant · 2021-05-20T19:03:14Z

We should run this again since there were changes

/test pull-cluster-api-provider-azure-upstream-v1alpha4-windows

jsturtevant · 2021-05-20T20:33:54Z

I do not think the failure was not related to these changes:

I0520 19:56:58.101565       1 azurejson_machine_controller.go:116] controllers/AzureJSONMachine "msg"="object was not found" "azureMachine"="capz-conf-lhu7ze-control-plane-262fp" "namespace"="capz-conf-lhu7ze" 
E0520 19:56:58.103259       1 controller.go:302] controller-runtime/manager/controller/azuremachine "msg"="Reconciler error" "error"="azuremachines.infrastructure.cluster.x-k8s.io \"capz-conf-lhu7ze-control-plane-262fp\" not found" "name"="capz-conf-lhu7ze-control-plane-262fp" "namespace"="capz-conf-lhu7ze" "reconciler group"="infrastructure.cluster.x-k8s.io" "reconciler kind"="AzureMachine"

/test pull-cluster-api-provider-azure-upstream-v1alpha4-windows

jsturtevant · 2021-05-20T21:41:10Z

The management cluster didn't come online, error was:

  �[91mTimed out after 180.003s.
  Deployment capi-system/capi-controller-manager failed to get status.Available = True condition

/test pull-cluster-api-provider-azure-upstream-v1alpha4-windows

devigned

lgtm
/assign @nader-ziada

CecileRobertMichon

/lgtm

CecileRobertMichon · 2021-05-25T22:51:56Z

/approve

k8s-ci-robot · 2021-05-25T22:52:06Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [CecileRobertMichon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot requested review from CecileRobertMichon and nader-ziada January 13, 2021 20:50

k8s-ci-robot added the sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. label Jan 13, 2021

jsturtevant mentioned this pull request Jan 13, 2021

🌱 Add optional parameter for using repolist to the kubetest library kubernetes-sigs/cluster-api#4069

Merged

jsturtevant force-pushed the windows-upstream-e2e branch from fbe4ed5 to 247ec83 Compare January 21, 2021 19:46

jsturtevant mentioned this pull request Jan 21, 2021

🌱 Cherry-pick: Add optional parameter for using repolist kubernetes-sigs/cluster-api#4102

Merged

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 28, 2021

jsturtevant mentioned this pull request Jan 28, 2021

kubeadm: write "cgroupfs" in the kubelet config kubernetes-sigs/sig-windows-tools#140

Closed

3 tasks

jsturtevant mentioned this pull request Mar 18, 2021

Track work needed to migrate upstream Windows tests from aks-engine to CAPZ Azure/container-upstream#98

Closed

17 tasks

jsturtevant force-pushed the windows-upstream-e2e branch from 247ec83 to e95e75e Compare March 26, 2021 18:32

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 26, 2021

jsturtevant marked this pull request as ready for review March 26, 2021 18:33

jsturtevant changed the title ~~[WIP] Windows upstream e2e~~ Windows upstream e2e Mar 26, 2021

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 26, 2021

jsturtevant mentioned this pull request Mar 26, 2021

Add windows upstream test job for capz kubernetes/test-infra#21544

Closed

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Mar 26, 2021

jsturtevant mentioned this pull request Mar 26, 2021

Add ability to inject kube binaries from kubernetes CI builds on Windows #1276

Closed

jsturtevant mentioned this pull request Apr 9, 2021

Add windows upstream e2e pr job kubernetes/test-infra#21748

Merged

devigned suggested changes May 11, 2021

View reviewed changes

jsturtevant mentioned this pull request May 14, 2021

Update number of ginkgo nodes for windows capz e2e kubernetes/test-infra#22180

Merged

jsturtevant force-pushed the windows-upstream-e2e branch 2 times, most recently from 155a07f to 142dc4a Compare May 14, 2021 02:41

CecileRobertMichon reviewed May 20, 2021

View reviewed changes

jsturtevant force-pushed the windows-upstream-e2e branch 2 times, most recently from dbd6d2a to 3537f83 Compare May 20, 2021 17:33

Upstream e2e tests for windows

3cc5960

jsturtevant force-pushed the windows-upstream-e2e branch from 3537f83 to 3cc5960 Compare May 20, 2021 17:37

devigned reviewed May 20, 2021

View reviewed changes

devigned approved these changes May 21, 2021

View reviewed changes

CecileRobertMichon reviewed May 24, 2021

View reviewed changes

k8s-ci-robot assigned CecileRobertMichon May 24, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 24, 2021

jsturtevant mentioned this pull request May 25, 2021

Inject Windows custom binaries for use in PRs and running against Kubernetes CI #1388

Merged

3 tasks

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 25, 2021

k8s-ci-robot merged commit 8ebf049 into kubernetes-sigs:master May 25, 2021

k8s-ci-robot added this to the v0.5.0 milestone May 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Windows upstream e2e #1119

Windows upstream e2e #1119

jsturtevant commented Jan 13, 2021 •

edited

Loading

k8s-ci-robot commented Jan 13, 2021

k8s-ci-robot commented Jan 13, 2021

jsturtevant commented Mar 26, 2021

CecileRobertMichon commented Mar 26, 2021

jsturtevant commented Mar 29, 2021

jsturtevant commented Apr 9, 2021

jsturtevant commented Apr 10, 2021

devigned left a comment

devigned May 11, 2021

jsturtevant May 14, 2021

jsturtevant commented May 14, 2021

nader-ziada commented May 14, 2021

jsturtevant commented May 14, 2021

jsturtevant commented May 14, 2021

jsturtevant commented May 20, 2021

CecileRobertMichon May 20, 2021

jsturtevant May 20, 2021 •

edited

Loading

jsturtevant May 20, 2021

CecileRobertMichon left a comment

jsturtevant commented May 20, 2021

devigned left a comment

devigned May 20, 2021

jsturtevant May 20, 2021

jsturtevant commented May 20, 2021

jsturtevant commented May 20, 2021

jsturtevant commented May 20, 2021

devigned left a comment

CecileRobertMichon left a comment

CecileRobertMichon commented May 25, 2021

k8s-ci-robot commented May 25, 2021

Windows upstream e2e #1119

Windows upstream e2e #1119

Conversation

jsturtevant commented Jan 13, 2021 • edited Loading

k8s-ci-robot commented Jan 13, 2021

k8s-ci-robot commented Jan 13, 2021

jsturtevant commented Mar 26, 2021

CecileRobertMichon commented Mar 26, 2021

jsturtevant commented Mar 29, 2021

jsturtevant commented Apr 9, 2021

jsturtevant commented Apr 10, 2021

devigned left a comment

Choose a reason for hiding this comment

devigned May 11, 2021

Choose a reason for hiding this comment

jsturtevant May 14, 2021

Choose a reason for hiding this comment

jsturtevant commented May 14, 2021

nader-ziada commented May 14, 2021

jsturtevant commented May 14, 2021

jsturtevant commented May 14, 2021

jsturtevant commented May 20, 2021

CecileRobertMichon May 20, 2021

Choose a reason for hiding this comment

jsturtevant May 20, 2021 • edited Loading

Choose a reason for hiding this comment

jsturtevant May 20, 2021

Choose a reason for hiding this comment

CecileRobertMichon left a comment

Choose a reason for hiding this comment

jsturtevant commented May 20, 2021

devigned left a comment

Choose a reason for hiding this comment

devigned May 20, 2021

Choose a reason for hiding this comment

jsturtevant May 20, 2021

Choose a reason for hiding this comment

jsturtevant commented May 20, 2021

jsturtevant commented May 20, 2021

jsturtevant commented May 20, 2021

devigned left a comment

Choose a reason for hiding this comment

CecileRobertMichon left a comment

Choose a reason for hiding this comment

CecileRobertMichon commented May 25, 2021

k8s-ci-robot commented May 25, 2021

jsturtevant commented Jan 13, 2021 •

edited

Loading

jsturtevant May 20, 2021 •

edited

Loading