This third-party package implements a Kubernetes-SIGs/kubetest2 tester capable of executing tests written using the e2e-framework.
First, you must ensure that you have the kubetest2 binary installed in the environment where you will run your tests :
go install sigs.k8s.io/kubetest2/...@latest
See further detail here.
To build the e2e-framework tester, you can use the following go install command:
go install sigs.k8s.io/e2e-framework/third_party/kubetest2/...@latest
Or, you can build from source:
go build ./cmd/kubetest2-tester-e2e-framework/
Either approach will produce a binary file named kubetest2-tester-e2e-framework, which follows a naming convention that the kubetest2 binary will use to launch the e2e-framework tester. Next, add your built e2e-framework tester binary to your operating system's PATH.
If the installation steps are successful, run the following to get information on how to run your tests with kubetest2 and the e2e-framewro:
$> kubetest2-tester-e2e-framework --help
Usage: kubetest2-tester-e2e-framework [e2e-framework-flags]
When used with kubetest2: kubetest2 [<deployer>] --test=e2e-framework -- [e2e-framework-flags]
e2e-framework flags:
--assess string Regular expression to select assessment(s) to run
--disable-graceful-teardown Ignore panic recovery while running tests. This will prevent test finish steps from getting executed on panic
--dry-run Run Test suite in dry-run mode. This will list the tests to be executed without actually running them
--fail-fast Fail immediately and stop running untested code
--feature string Regular expression to select feature(s) to test
-h, --help
--kubeconfig string Path to a cluster kubeconfig file (optional)
--labels string Comma-separated key=value to filter features by labels
--namespace string A namespace value to use for testing (optional)
--packages string Space-separated packages to test
--parallel Run test features in parallel
--skip-assessments string Regular expression to skip assessment(s) to run
--skip-features string Regular expression to skip feature(s) to run
--skip-labels string Regular expression to skip label(s) to run
--test-flags string Space-separated flags applied to 'go test' command
To run a test with kubetest2, you must follow this command format as outlined above:
kubetest2 [<deployer>] --test=e2e-framework -- [e2e-framework-flags]
Where:
deployeris a binary used to stand up an infrastructure (not provided here).--test=e2e-frameworkspecifies the tester to used, in this case the e2e-framework tester.e2e-framework-flagsare the list of CLI flags that are passed to the e2e-framework binary.
Let us use kubetest2 to run a simple test with no deployer and no arguments passed to the test:
kubetest2 noop --test=e2e-framework -- --packages ./examples/simple
The previous command will launch kubetest2 without starting a cluster (noop deployer). Note the -- which indicates the start of arguments which will be passed to the tester (e2e-framework).
If the command is successful, you will see the kubetest2 logs similar to what is shown:
I1211 08:56:43.836568 14380 app.go:61] The files in RunDir shall not be part of Artifacts
I1211 08:56:43.837144 14380 app.go:62] pass rundir-in-artifacts flag True for RunDir to be part of Artifacts
I1211 08:56:43.837188 14380 app.go:64] RunDir for this run: "/home/ubuntu/go/e2e-framework/examples/_rundir/cf42e94d-1c2e-4c6f-9912-2ff948608008"
I1211 08:56:43.860870 14380 app.go:128] ID for this run: "cf42e94d-1c2e-4c6f-9912-2ff948608008"
Running: go test ./simple -args
ok sigs.k8s.io/e2e-framework/examples/simple 0.363s
You can easily change this command to stand up and teardown a Kubernetes cluster using KinD for instance (kubetest2 also supports other deployers):
$> kubetest2 kind --up --down --test=e2e-framework -- --packages .
I1211 09:20:46.640997 19579 up.go:62] Up(): creating kind cluster...
Creating cluster "kind" ...
• Ensuring node image (kindest/node:v1.25.3) 🖼 ...
✓ Ensuring node image (kindest/node:v1.25.3) 🖼
• Preparing nodes 📦 ...
...
Running: go test -args
PASS
ok sigs.k8s.io/e2e-framework/kubetest2test 0.307s
I1211 09:23:14.406049 19579 down.go:32] Down(): deleting kind cluster...
Deleting cluster "" ...
It should be noted that e2e-framework offers its own programmatic hooks for more flexible controls of pre and post test tasks, such as creating and tearing down named clusters (see the examples folder).
The kubetest2 e2e-framework tester supports the ability to pass CLI arguments to control settings such as test execution filters and test behaviors. Recall, in order for your e2e-framework tests to receive CLI arguments, you must programmatically create the test environment to receive its configuration from the command-line:
var tenv env.Environment
func TestMain(m *testing.M) {
var err error
tenv, err = env.NewFromFlags()
if err != nil {
log.Fatalf("failed to build env from flags: %s", err)
}
os.Exit(tenv.Run(m))
}
func TestClusterObjects(t *testing.T) {
f := features.New("cluster-test")
f.Assess("pod-count", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
var pods corev1.PodList
client, err := cfg.NewClient()
if err != nil {
t.Fatal(err)
}
err = client.Resources("kube-system").List(context.TODO(), &pods)
if err != nil {
t.Fatal(err)
}
if len(pods.Items) == 0 {
t.Fatal("no pods in namespace kube-system")
}
return ctx
})
f.Assess("dep-count", func(ctx context.Context, t *testing.T, cfg *envconf.Config) context.Context {
var deps appsv1.DeploymentList
client, err := cfg.NewClient()
if err != nil {
t.Fatal(err)
}
err = client.Resources().List(context.TODO(), &deps)
if err != nil {
t.Fatal(err)
}
if len(deps.Items) == 0 {
t.Fatal("deployments not found")
}
return ctx
})
tenv.Test(t, f.Feature())
}This test can be launched with kubetest2 to include additional CLI flags that are passed to the e2e-framework tester for configurations, as show:
$> kubetest2 kind --up --down \
--test=e2e-framework -- \
--packages ./cluster \
--kubeconfig=$HOME/.kube/config
The previous command will stand up a kind cluster and pass additional CLI flags --packages to indicate the test package and and --kubeconfig to point to the configuration file.
The test can be launched to only run the dep-count assessment by skipping the others, as shown in the following comomand:
$> kubetest2 kind --up --down \
--test=e2e-framework -- \
--packages ./cluster \
--kubeconfig=$HOME/.kube/config \
--skip-assessments=pod-count
In case you want to pass additional CLI flags to the Go test binary itself, you can use flag --test-flags to do so. For instance, the following runs the test with verbose output:
$> kubetest2 kind --up --down \
--test=e2e-framework -- \
--packages ./cluster \
--kubeconfig=$HOME/.kube/config \
--skip-assessments=pod-count
--test-flags="-v"
The previous kubetest2 command will run the following go test command as shown:
$> go test -v ./cluster -args --kubeconfig=/Users/vivienv/.kube/config --skip-assessment=pod-count
=== RUN TestClusterObjects
=== RUN TestClusterObjects/cluster-test
=== RUN TestClusterObjects/cluster-test/pod-count
env.go:441: Skipping assessment: "pod-count": name matched
=== RUN TestClusterObjects/cluster-test/dep-count
--- PASS: TestClusterObjects (0.02s)
--- PASS: TestClusterObjects/cluster-test (0.02s)
--- SKIP: TestClusterObjects/cluster-test/pod-count (0.00s)
--- PASS: TestClusterObjects/cluster-test/dep-count (0.02s)
PASS
ok sigs.k8s.io/e2e-framework/kubetest2test/cluster 0.374s