diff --git a/docs/development_guide.md b/docs/development_guide.md index 17d91dc9d8..cf55f950fa 100644 --- a/docs/development_guide.md +++ b/docs/development_guide.md @@ -130,6 +130,100 @@ golang.org/x/tools/cmd/goimports`. It's recommended that you set your IDE or other development tools to use `goimports`. Formatting is checked during CI by the `bin/fmt` script. +## Testing your changes + +The OSM repo has a few layers of tests: + - unit tests + - integration tests + - simulations + +For tests in this repo we have chosen to leverage the +[Gomega](https://onsi.github.io/gomega/) and +[Ginkgo](https://onsi.github.io/ginkgo/) frameworks. We follow Go's convention and add +unit tests for the respective functions in files with the `_test.go` suffix. So if a +function lives in a file `foo.go` we will write a test for it in the file `foo_test.go`. +For more about Go testing +read [the following document](https://tip.golang.org/cmd/go/#hdr-Test_packages). + +Take a look at any of the [existing unit-test examples](https://github.com/openservicemesh/osm/blob/release-v0.2/pkg/catalog/catalog_test.go) +should you need a starting point. + +Often times we add a [suite_test.go](https://github.com/openservicemesh/osm/blob/release-v0.2/pkg/catalog/suite_test.go) +file, which serves as an entry point for the Ginkgo tests within the given package. + +#### Unit Tests +The most rudimentary tests are the unit tests. We strive for test coverage above 80% where +this is pragmatic and possible. +Each newly added function should be accompanied by a unit test. Ideally, while working +on this repository, we practice +[test-driven development](https://en.wikipedia.org/wiki/Test-driven_development), +and each change would be accompanied by a unit test. + +To run all unit tests you can use the following `Makefile` target: +```bash +make go-tests +``` + +You can run the tests exclusively for the package you are working on. For example the following command will +run only the tests in the package implementing OSM's +[Hashicorp Vault](https://www.vaultproject.io/) integration: +```bash +go test ./pkg/certificate/providers/vault/... +``` + +You can check the unit test coverage by using the `-cover` option: +```bash +go test -cover ./pkg/certificate/providers/vault/... +``` + +We have a dedicated tool for in-depth analysis of the unit-test code coverage: +```bash +./scripts/test-w-coverage.sh +``` +Running the [test-w-coverage.sh](../scripts/test-w-coverage.sh) script will create +an HTML file with in-depth analysis of unit-test coverage per package, per +function, and it will even show lines of code that need work. Open the HTML +file this tool generates to understand how to improve test coverage: +``` +open ./coverage/index.html +``` + +Once the file loads in your browser, scroll to the package you worked on to see current test coverage: + + + +Our overall guiding principle is to maintain unit-test coverage at or above 80%. + +To understand which particular functions need more testing - scroll further in the report: + + + +And if you are wondering why a function, which we have written a test for, is not 100% covered, +you will find the per-function analysis useful. This will show you code paths that are not tested. + + + +#### Integration Tests + +Unit tests focus on a single function. These ensure that with a specific input, the function +in question produces expected output or side effect. Integration tests, on the other hand, +ensure that multiple functions work together correctly. Integration tests ensure your new +code composes with other existing pieces. + +Take a look at [the following test](https://github.com/openservicemesh/osm/blob/release-v0.2/pkg/configurator/client_test.go), +which tests the functionality of multiple functions together. In this particular example, the test: + - uses a mock Kubernetes client via `testclient.NewSimpleClientset()` from the `k8s.io/client-go/kubernetes/fake` library + - [creates a ConfigMap](https://github.com/openservicemesh/osm/blob/release-v0.2/pkg/configurator/client_test.go#L32) + - [tests whether](https://github.com/openservicemesh/osm/blob/release-v0.2/pkg/configurator/client_test.go#L95-L96) the underlying functions compose correctly by fetching the results of the top-level function `GetMeshCIDRRanges()` + + +#### Simulation / Demo +When we want to ensure that the entire system works correctly over time and +transitions state as expected - we run +[the demo included in the docs](https://github.com/openservicemesh/osm/blob/main/docs/example/README.md). +This type of test is the slowest, but also most comprehensive. This test will ensure that your changes +work with a real Kubernetes cluster, with real SMI policy, and real functions - no mocked or fake Go objects. + ## Helm charts The Open Service Mesh control plane chart is located in the diff --git a/docs/images/unit-test-coverage-1.png b/docs/images/unit-test-coverage-1.png new file mode 100644 index 0000000000..356cb5a5bb Binary files /dev/null and b/docs/images/unit-test-coverage-1.png differ diff --git a/docs/images/unit-test-coverage-2.png b/docs/images/unit-test-coverage-2.png new file mode 100644 index 0000000000..50663aa955 Binary files /dev/null and b/docs/images/unit-test-coverage-2.png differ diff --git a/docs/images/unit-test-coverage-3.png b/docs/images/unit-test-coverage-3.png new file mode 100644 index 0000000000..8637595fef Binary files /dev/null and b/docs/images/unit-test-coverage-3.png differ