Skip to content

Commit

Permalink
Move docs from knative/serving to knative/pkg (#2097)
Browse files Browse the repository at this point in the history 
* Move docs from knative/serving to knative/pkg

While working on tektoncd/pipeline#16
I started trying to build some integration tests, and I wanted to use
the code which had been moved to knative/pkg, but it turned out the docs
hadn't been ported with the code, so I am moving the docs relevant to
knative/pkg/test to that repo.

I also fixed some headings which were duplicated.

Adding docs to knative/pkg in knative/pkg#102

* Fix name of function in docs

lol this one was wrong THE WHOLE TIME SINCE THE VERY FIRST COMMIT where
I used one name in the implementation and a different name in the docs
XD 5bc7377
  • Loading branch information
@bobcatfish @knative-prow-robot
bobcatfish authored and knative-prow-robot committed Oct 4, 2018
1 parent e17bfde commit 78eba97
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 154 deletions.
74 changes: 5 additions & 69 deletions test/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ These tests require:
```
3. A docker repo containing [the test images](#test-images)

### Flags
### Common Flags

* By default the e2e tests against the current cluster in `~/.kube/config`
using the environment specified in [your environment variables](/DEVELOPMENT.md#environment-setup).
Expand Down Expand Up @@ -115,6 +115,7 @@ To run the script for all end to end test images:
```bash
./test/upload-test-images.sh
```

A docker tag may be passed as an optional parameter. This can be
useful on [Minikube] in tandem with the `--tag` [flag](#using-a-docker-tag):

Expand All @@ -132,58 +133,12 @@ New test images should be placed in `./test/test_images`.
These flags are useful for running against an existing cluster, making use of your existing
[environment setup](/DEVELOPMENT.md#environment-setup).

Tests importing [`github.com/knative/serving/test`](adding_tests.md#test-library) recognize these flags:
Tests importing [`github.com/knative/serving/test`](#test-library) recognize these flags:

* [`--kubeconfig`](#specifying-kubeconfig)
* [`--cluster`](#specifying-cluster)
* [`--namespace`](#specifying-namespace)
* [All flags added by `knative/pkg/test`](https://github.com/knative/pkg/tree/master/test#flags)
* [`--dockerrepo`](#overriding-docker-repo)
* [`--tag`](#using-a-docker-tag)
* [`--resolvabledomain`](#using-a-resolvable-domain)
* [`--logverbose`](#output-verbose-logs)
* [`--emitmetrics`](#emit-metrics)

### Specifying kubeconfig

By default the tests will use the [kubeconfig
file](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/)
at `~/.kube/config`. If there is an error getting the current user, it will use `kubeconfig` instead as the default value.
You can specify a different config file with the argument `--kubeconfig`.

To run the tests with a non-default kubeconfig file:

```bash
go test -v -tags=e2e -count=1 ./test/conformance --kubeconfig /my/path/kubeconfig
go test -v -tags=e2e -count=1 ./test/e2e --kubeconfig /my/path/kubeconfig
```

### Specifying cluster

The `--cluster` argument lets you use a different cluster than [your specified
kubeconfig's](#specifying-kubeconfig) active context. This will default to the value
of your [`K8S_CLUSTER_OVERRIDE` environment variable](/DEVELOPMENT.md#environment-setup)
if not specified.
```bash
go test -v -tags=e2e -count=1 ./test/conformance --cluster your-cluster-name
go test -v -tags=e2e -count=1 ./test/e2e --cluster your-cluster-name
```
The current cluster names can be obtained by running:
```bash
kubectl config get-clusters
```
### Specifying namespace
The `--namespace` argument lets you specify the namespace to use for the
tests. By default, tests will use `serving-tests`.
```bash
go test -v -tags=e2e -count=1 ./test/conformance --namespace your-namespace-name
go test -v -tags=e2e -count=1 ./test/e2e --namespace your-namespace-name
```

### Overridding docker repo

Expand Down Expand Up @@ -212,7 +167,6 @@ go test -v -tags=e2e -count=1 ./test/e2e --tag any-old-tag
Of course, this implies that you tagged the images when you [uploaded
them](#building-the-test-images).

### Using a resolvable domain

If you set up your cluster using [the getting started
Expand All @@ -226,22 +180,4 @@ the header.

If you have configured your cluster to use a resolvable domain, you can use the
`--resolvabledomain` flag to indicate that the test should make requests directly against
`Route.Status.Domain` and does not need to spoof the `Host`.
### Output verbose logs
The `--logverbose` argument lets you see verbose test logs and k8s logs.
```bash
go test -v -tags=e2e -count=1 ./test/e2e --logverbose
```
### Emit metrics
Running tests with the `--emitmetrics` argument will cause latency metrics to be emitted by
the tests.
* To add additional metrics to a test, see [emitting metrics](adding_tests.md#emit-metrics).
* For more info on the format of the metrics, see [metric format](adding_tests.md#metric-format).
[Minikube]: https://kubernetes.io/docs/setup/minikube/
`Route.Status.Domain` and does not need to spoof the `Host`.
97 changes: 12 additions & 85 deletions test/adding_tests.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,23 @@ so that `go test ./...` can be used to run only [the unit tests](README.md#runni
In the [`test`](/test/) dir you will find several libraries in the `test` package
you can use in your tests.

You can:
This library exists partially in this directory and partially in
[`knative/pkg/test`](https://github.com/knative/pkg/tree/master/test).

The libs in this dir can:

* [Use common test flags](#use-common-test-flags)
* [Output logs](#output-logs)
* [Emit metrics](#emit-metrics)
* [Get access to client objects](#get-access-to-client-objects)
* [Make requests against deployed services](#make-requests-against-deployed-services)
* [Check Knative Serving resources](#check-knative-serving-resources)
* [Verify resource state transitions](#verify-resource-state-transitions)
* [Generate boilerplate CRDs](#generate-boilerplate-crds)
* [Ensure test cleanup](#ensure-test-cleanup)

See [`knative/pkg/test`](https://github.com/knative/pkg/tree/master/test) to:

* Output logs
* Emit metrics
* Ensure test cleanup

### Use common test flags

Expand All @@ -47,73 +53,6 @@ imagePath := strings.Join([]string{test.ServingFlags.DockerRepo, image}, "/"))

_See [e2e_flags.go](./e2e_flags.go)._

### Output logs

[When tests are run with `--logverbose`
option](README.md#output-verbose-logs), debug logs will be emitted to stdout.

We are using the common [e2e logging library](/pkg/test/logging) that uses the [Knative logging library](/pkg/logging/) for structured logging.
It is built on top of [zap](https://github.com/uber-go/zap). Tests should initialize the global logger to use a test specifc context with `logging.GetContextLogger`:

```go
// The convention is for the name of the logger to match the name of the test.
logging.GetContextLogger("TestHelloWorld")
```

Logs can then be emitted using the `logger` object which is required by
many functions in the test library. To emit logs:

```go
logger.Infof("Creating a new Route %s and Configuration %s", route, configuration)
logger.Debugf("The LogURL is %s, not yet verifying", logURL)
```

_See [logging.go](./logging/logging.go)._

### Emit metrics

You can emit metrics from your tests using [the opencensus
library](https://github.com/census-instrumentation/opencensus-go), which [is being
used inside Knative as well](/docs/telemetry.md). These metrics will be emitted by
the test if the test is run with [the `--emitmetrics` option](README.md#emit-metrics).

You can record arbitrary metrics with
[`stats.Record`](https://github.com/census-instrumentation/opencensus-go#stats) or
measure latency with [`trace.StartSpan`](https://github.com/census-instrumentation/opencensus-go#traces):

```go
ctx, span := trace.StartSpan(context.Background(), "MyMetric")
```

* These traces will be emitted automatically by [the generic crd polling
functions](#check-knative-serving-resources).
* The traces are emitted by [a custom metric exporter](./logging.go)
that uses the global logger instance.

#### Metric format

When a `trace` metric is emitted, the format is `metric name startTime endTime duration`. The name
of the metric is arbitrary and can be any string. The values are:

* "metric" - Indicates this log is a metric
* name - Arbitrary string indentifying the metric. TODO(#1379) determine metric format
* startTime - Unix time in nanoseconds when measurement started
* endTime - Unix time in nanoseconds when measurement ended
* duration - The difference in ms between the startTime and endTime

For example:

```bash
metric WaitForConfigurationState/prodxiparjxt/ConfigurationUpdatedWithRevision 1529980772357637397 1529980772431586609 73.949212ms
```

_The [`Wait` methods](#check-knative-serving-resources) (which poll resources) will
prefix the metric names with the name of the function, and if applicable, the name of the resource,
separated by `/`. In the example above, `WaitForConfigurationState` is the name of
the function, and `prodxiparjxt` is the name of the configuration resource being polled.
`ConfigurationUpdatedWithRevision` is the string passed to `WaitForConfigurationState` by
the caller to identify what state is being polled for._

### Get access to client objects

To initialize client objects that you can use [the command line flags](#use-flags)
Expand Down Expand Up @@ -275,15 +214,15 @@ _See [states.go](./states.go)._
Your tests will probably need to create `Route` and `Configuration` objects. You can use the
existing boilerplate to describe them.

You can also use the function `RandomizedName` to create a random name for your `crd` so that
You can also use the function `AppendRandomString` to create a random name for your `crd` so that
your tests can use unique names each time they run.

For example to create a `Configuration` object that uses a certain docker image with a
randomized name:

```go
var names test.ResourceNames
names.Config := test.RandomizedName('hotdog')
names.Config := test.AppendRandomString('hotdog', logger)
_, err := clients.ServingClient.Create(test.Configuration(namespaceName, names, imagePath))
if err != nil {
return err
Expand All @@ -293,15 +232,3 @@ if err != nil {
Please expand these functions as more use cases are tested.

_See [crd.go](./crd.go)._

### Ensure test cleanup

To ensure your test is cleaned up, you should defer cleanup to execute after your
test completes and also ensure the cleanup occurs if the test is interrupted:

```go
defer tearDown(clients)
test.CleanupOnInterrupt(func() { tearDown(clients) })
```

_See [cleanup.go](./cleanup.go)._

0 comments on commit 78eba97

Please sign in to comment.