Run VSphere tests in self-hosted, refactor test suite into spec per provider, support label filtering, get all tests green #360
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
squizzi
added
the
test e2e
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
2 times, most recently
from
dc5f8d2
to
605513d
Compare
This comment was marked as resolved.
This comment was marked as resolved.
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
5 times, most recently
from
92cc4f5
to
d4f0344
Compare
This was referenced Sep 24, 2024
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
from
d4f0344
to
fea83d7
Compare
squizzi
changed the title
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
4 times, most recently
from
f76cfe0
to
e19f75f
Compare
squizzi
changed the title
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
8 times, most recently
from
7560f52
to
00088a6
Compare
|
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
2 times, most recently
from
dc6d505
to
6386214
Compare
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
3 times, most recently
from
6c6fc9f
to
25aef1d
Compare
|
Note: This run shows a vsphere green: https://github.com/Mirantis/hmc/actions/runs/11153080327/job/31036965900?pr=360 We might see some failures in upcoming runs as Mirantis IT is doing some work on our self-hosted runner. |
kylewuolle
requested changes
Oct 3, 2024
kylewuolle
previously approved these changes
Oct 3, 2024
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
from
4e915c7
to
672d5f2
Compare
|
csi-drivers are going to be the death of me 😂 |
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
2 times, most recently
from
f6cb0da
to
a65544d
Compare
|
The Deleting the machine triggers the From the CAPI code it appears were here ^^ but there's no log statements or events pertaining to not being able to find the Checking the code, it iterates the
|
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
from
a65544d
to
86dc992
Compare
* Break provider tests into seperate files with labels representing either onprem or cloud providers. * Add new jobs to CI workflow which dictate where tests will run, onprem provider tests like vSphere will run on self-hosted runners since they will use internal resources to test. Cloud provider tests will use the existing workflow since they can access providers without network access and can take advantage of the much larger GitHub hosted pool. Hosted/Self-hosted tests can run concurrently. * Make Cleanup depend on the cloud-e2etest only. * Use new GINKGO_LABEL_FILTER to manage what tests run where. * Move controller validatation into BeforeSuite since the controller needs to be up and ready for each provider test, this will also enable us to add controller specific test cases later and make those run without the "test e2e" flag. * Seperate self-hosted and hosted test concurrency groups * Update docs with test filtering instructions * Ensure a Release exists for the custom build.version we deploy * Move all e2e related helpers into e2e dir * Add new clusteridentity package for creating ClusterIdentity kind's and associated Secret's. * Merge PR workflows together * Make sure VERSION gets passed across jobs * Ensure uniqueness among deployed ManagedClusters, simplify MANAGED_CLUSTER_NAME in CI to prevent Azure availabilitySetName validation error. * Default Azure test templates to uswest2 to prevent issues with AvailabilityZone. * Use the same concurrency-group across all jobs, except Cleanup which intentionally does not belong to a concurrency-group. * Use Setup Go across jobs for caching. * Support patching other hosted clusters to status.ready with a common patching helper. * Move VSphere delete into AfterEach to serve as cleanup. * Add support for cleaning Azure resources. * Prevent ginkgo from timing out tests. * Use azure-disk CSI driver we deploy via templates. Signed-off-by: Kyle Squizzato <ksquizzato@mirantis.com>
squizzi
force-pushed
the
vsphere-tests-on-self-hosted
branch
from
86dc992
to
3a4c09c
Compare
kylewuolle
approved these changes
Oct 9, 2024
slysunkin
approved these changes
Oct 10, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors our existing e2e test spec into a more conventional e2e test run. We now have a
BeforeSuitethat runs and installs/validates the controller. Prior to this we had all of our e2e tests under a singleControllerspec, but this breaks it into:Label(controller))Label(provider))Label(provider:aws, provider:cloud)) -test/e2e/provider_aws_test.goLabel(provider:azure, provider:cloud)) -test/e2e/provider_azure_test.goLabel(provider:vsphere, provider:onprem)) -test/e2e/provider_vsphere_test.goThe labels for these tests have been documented here: https://github.com/Mirantis/hmc/pull/360/files#diff-3a6cab76ecce19612a704fe0ef89dc83f1ed2df44f7bfa28d47d2529a7949fafR139
Each of the umbrella labels correspond to a GitHub job that is dependent on the
Build and Unit Testjob which provides setup. TheControllertests always run, no matter the label, right now theControllertest is disabled because there aren't any tests other than Before/AfterSuite, but we can enable it because that sort of is a test of the controller in itself.The provider tests are still attached to the
test e2elabel, but I think we can break these out further withproviderspecific GitHub labels at some point.provider:onpremtests run on a self-hosted GitHub runner that has access to Mirantis' network, this is to support providers that do not necessarily have cloud infrastructure we can get to from a GitHub hosted runner -- like VSphere.This PR also adds support for creating
ClusterIdentity,Credentialand their associatedSecretresources with a newclusteridentitypackage.And of course, with testing comes bug fixing, so this PR fixes several different issues discovered along the way.
Closes: #210, Closes: #323
Note: I've opened two other issues to further simplify these tests, we could make these a lot easier to write in the future with: