From 8b6a862f5331669e977530d83427a980273b6e54 Mon Sep 17 00:00:00 2001 From: Rebecca Zanzig Date: Mon, 5 Nov 2018 16:27:06 -0800 Subject: [PATCH 1/3] Add information about writing bats unit tests This documentation gives a description of how the bats tests are structured, as well as giving several examples of common test patterns. --- README.md | 108 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 108 insertions(+) diff --git a/README.md b/README.md index 0eb6ca0cf965..181a44a518d1 100644 --- a/README.md +++ b/README.md @@ -66,3 +66,111 @@ that can be used to quickly bring up a GKE cluster and configure `kubectl` and `helm` locally. This can be used to quickly spin up a test cluster for acceptance tests. Unit tests _do not_ require a running Kubernetes cluster. + +### Writing Unit Tests + +Changes to the Helm chart should be accompanied by appropriate unit tests. + +#### Formatting + +- Put tests in the test file in the same order as the variables appear in the `values.yaml`. +- Start tests for a chart value with a header that says what is being tested, like this: + ``` + #-------------------------------------------------------------------- + # annotations + ``` + +- Name the test based on what it's testing in the following format: + ``` + { section being tested }: { short description of the test case } + ``` + + When adding tests to an existing file, the first section will be the same as the other tests in the file. + +#### Test Details + +[Bats](https://github.com/bats-core/bats-core) provides a way to run commands in a shell and inspect the output in an automated way. +In all of the tests in this repo, the base command being run is [helm template](https://docs.helm.sh/helm/#helm-template) which turns the templated files into straight yaml output. +In this way, we're able to test that the various conditionals in the templates render as we would expect. + +Each test defines the files that should be rendered using the `-x` flag, then it might adjust chart values by adding `--set` flags as well. +The output from this `helm template` command is then piped to [yq](https://pypi.org/project/yq/). +`yq` allows us to pull out just the information we're interested in, either by referencing its position in the yaml file directly or giving information about it (like its length). + +The test passes or fails based on the conditional at the end that is in square brackets, which is a comparison of our expected value and the output of `helm template` piped to `yq`. + +The `| tee /dev/stderr ` pieces direct any terminal output of the `helm template` and `yq` commands to stderr so that it doesn't interfere with `bats`. + +#### Test Examples + +Here are some examples of common test patterns: + +- Check that a value is disabled by default + + ``` + @test "ui/Service: no type by default" { + cd `chart_dir` + local actual=$(helm template \ + -x templates/ui-service.yaml \ + . | tee /dev/stderr | + yq -r '.spec.type' | tee /dev/stderr) + [ "${actual}" = "null" ] + } + ``` + + In this example, nothing is changed from the default templates (no `--set` flags), then we use `yq` to retrieve the value we're checking, `.spec.type`. + This output is then compared against our expected value (`null` in this case) in the assertion `[ "${actual}" = "null" ]` + + +- Check that a template value is rendered to a specific value + ``` + @test "ui/Service: specified type" { + cd `chart_dir` + local actual=$(helm template \ + -x templates/ui-service.yaml \ + --set 'ui.service.type=LoadBalancer' \ + . | tee /dev/stderr | + yq -r '.spec.type' | tee /dev/stderr) + [ "${actual}" = "LoadBalancer" ] + } + ``` + + This is very similar to the last example, except we've changed a default value with the `--set` flag and correspondingly changed the expected value. + + - Check that a template value contains several values + ``` + @test "syncCatalog/Deployment: to-k8s only" { + cd `chart_dir` + local actual=$(helm template \ + -x templates/sync-catalog-deployment.yaml \ + --set 'syncCatalog.enabled=true' \ + --set 'syncCatalog.toConsul=false' \ + . | tee /dev/stderr | + yq '.spec.template.spec.containers[0].command | any(contains("-to-consul=false"))' | tee /dev/stderr) + [ "${actual}" = "true" ] + + local actual=$(helm template \ + -x templates/sync-catalog-deployment.yaml \ + --set 'syncCatalog.enabled=true' \ + --set 'syncCatalog.toConsul=false' \ + . | tee /dev/stderr | + yq '.spec.template.spec.containers[0].command | any(contains("-to-k8s"))' | tee /dev/stderr) + [ "${actual}" = "false" ] + } + ``` + In this case, the same command is run twice in the same test. + This can be used to look for several things in the same field, or to check that something is not present that shouldn't be. + +- Check that an entire template file is not rendered + ``` + @test "syncCatalog/Deployment: disabled by default" { + cd `chart_dir` + local actual=$(helm template \ + -x templates/sync-catalog-deployment.yaml \ + . | tee /dev/stderr | + yq 'length > 0' | tee /dev/stderr) + [ "${actual}" = "false" ] + } + ``` + Here we are check the length of the command output to see if the anything is rendered. + This style can easily be switched to check that a file is rendered instead. From e8d041284eba585671b4c583277d1bf797c1b0b5 Mon Sep 17 00:00:00 2001 From: Rebecca Zanzig Date: Wed, 7 Nov 2018 11:58:07 -0800 Subject: [PATCH 2/3] Clarify the test naming guidelines Clarified the test naming by including the full first line pattern that includes the name. --- README.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 181a44a518d1..4f87fd32b27f 100644 --- a/README.md +++ b/README.md @@ -80,9 +80,9 @@ Changes to the Helm chart should be accompanied by appropriate unit tests. # annotations ``` -- Name the test based on what it's testing in the following format: +- Name the test based on what it's testing in the following format (this will be its first line): ``` - { section being tested }: { short description of the test case } + @test "
: " { ``` When adding tests to an existing file, the first section will be the same as the other tests in the file. @@ -95,7 +95,8 @@ In this way, we're able to test that the various conditionals in the templates r Each test defines the files that should be rendered using the `-x` flag, then it might adjust chart values by adding `--set` flags as well. The output from this `helm template` command is then piped to [yq](https://pypi.org/project/yq/). -`yq` allows us to pull out just the information we're interested in, either by referencing its position in the yaml file directly or giving information about it (like its length). +`yq` allows us to pull out just the information we're interested in, either by referencing its position in the yaml file directly or giving information about it (like its length). +The `-r` flag can be used with `yq` to return a raw string instead of a quoted one which is especially useful when looking for an exact match. The test passes or fails based on the conditional at the end that is in square brackets, which is a comparison of our expected value and the output of `helm template` piped to `yq`. @@ -119,7 +120,7 @@ Here are some examples of common test patterns: ``` In this example, nothing is changed from the default templates (no `--set` flags), then we use `yq` to retrieve the value we're checking, `.spec.type`. - This output is then compared against our expected value (`null` in this case) in the assertion `[ "${actual}" = "null" ]` + This output is then compared against our expected value (`null` in this case) in the assertion `[ "${actual}" = "null" ]`. - Check that a template value is rendered to a specific value @@ -137,7 +138,7 @@ Here are some examples of common test patterns: This is very similar to the last example, except we've changed a default value with the `--set` flag and correspondingly changed the expected value. - - Check that a template value contains several values +- Check that a template value contains several values ``` @test "syncCatalog/Deployment: to-k8s only" { cd `chart_dir` From 8c4cff9401322c2f17c63ca7ce276177ca8aa577 Mon Sep 17 00:00:00 2001 From: Rebecca Zanzig Date: Wed, 7 Nov 2018 12:16:24 -0800 Subject: [PATCH 3/3] Add note about testing more than two conditions in one test For testing two conditions in the same test, it is reasonable to follow the existing test patterns and run `helm template` each time. When testing more conditions, it starts to make sense to separate the rendering of the yaml from the condition finding and testing. This commit adds a note to that effect. --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 4f87fd32b27f..80686d025d47 100644 --- a/README.md +++ b/README.md @@ -162,6 +162,8 @@ Here are some examples of common test patterns: In this case, the same command is run twice in the same test. This can be used to look for several things in the same field, or to check that something is not present that shouldn't be. + *Note:* If testing more than two conditions, it would be good to separate the `helm template` part of the command from the `yq` sections to reduce redundant work. + - Check that an entire template file is not rendered ``` @test "syncCatalog/Deployment: disabled by default" {