feat: new dev/test environment #414
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apricote
reviewed
Apr 6, 2023
apricote
reviewed
Apr 6, 2023
tests/e2e/e2e_test.go
Outdated
| WaitForHTTPOnServer(t, testCluster.setup.ExtServer, testCluster.setup.privKey, pod.Status.PodIP, false) | ||
|
|
||
| nwTest.TearDown() | ||
| network, _, err := testCluster.hcloud.Network.Get(context.TODO(), testCluster.scope) |
There was a problem hiding this comment.
Depending on the timing and execution order of the tests the hccm might not have setup the routes yet. To avoid any flaky tests we should poll for this condition.
Might be something for a follow up PR though.
apricote
reviewed
Apr 6, 2023
apricote
reviewed
Apr 6, 2023
Co-authored-by: Julian Tölle <julian.toelle@hetzner-cloud.de>
Co-authored-by: Julian Tölle <julian.toelle@hetzner-cloud.de>
Co-authored-by: Julian Tölle <julian.toelle@hetzner-cloud.de>
apricote
approved these changes
Apr 6, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR simplifies and unifies the process of bringing up a hcloud k8s environment that is suitable for development/testing purposes. It builds off the work originally started in hetznercloud/csi-driver#226
The central script is
hack/dev-up.sh. The script requires aHCLOUD_TOKEN, and will build a 1+ k8s cluster usingk3sup+hcloudCLI tools. It's typically quite fast: a cold execution of the script should take less <1min before a fully operational cluster is ready for use.The
dev-down.shscript will delete all resources created bydev-up.sh.One of the primary objectives in this work was to simplify the e2e testing process. Julian and I decided to remove the matrix of kubeadm clusters, and also remove flannel as an option for CNI. Instead, our test suite now runs on the latest 3 stable releases of k3s. Further, the option to create a cluster with/without hcloud networking support has been removed. Created clusters always have a network and an install of Cilium configured for direct routing mode (traffic bound for pods on other nodes is handed to hcloud network fabric).
Skaffold is plumbed into the e2e test pipelines on GitHub. This way, the process of building and deploying a test build of hccm to a cluster is unified for development and test.
Once this work has landed and stabilized here, we expect to package it up a bit further and use it in csi-driver and anywhere else we operate Kubernetes integrations that need automated testing.