Add skeleton integration test

In this first step, we add an empty integration test that does nothing
but list `Pipeline`s in a namespace harded coded to `arendelle`
(already used the Toy Story theme in knative/serving, needed something
different here, thanks @aaron-prindle XD).

This can be run manually against a developer's own cluster, so folks can
start adding integration tests in parallel if they want to.

This is based off the end to end tests in knative/serving
(https://github.com/knative/serving/tree/master/test#running-end-to-end-tests)
and uses the libs in knative/pkg extensively, which were made from the
libs in knative/serving. The docs weren't moved with the libs tho which
I'm trying to fix in:
* knative/serving#2097
* knative/pkg#102

The commit adds opencensus, since this is used by the test libs for
collecting and emitting metrics. We might want to use those eventually
too so that seems fine tho it would be nice if they were optional
¯\_(ツ)_/¯

This is the first step for #16, next I'll be looking into executing these
via Prow, either against a hardcoded cluster with the secrets in Prow or
going straight for kubetest + boskos.

DEVELOPMENT.md

@@ -74,7 +74,7 @@ for your `KO_DOCKER_REPO` if required. To be able to push images to `gcr.io/<pro
 gcloud auth configure-docker
 ```
 
-4. The user you are using to interact with your k8s cluster must be a cluster admin to create role bindings:
+The user you are using to interact with your k8s cluster must be a cluster admin to create role bindings:
 
 ```shell
 # Using gcloud to get your current user
@@ -94,6 +94,7 @@ While iterating on the project, you may need to:
 1. Update your (external) dependencies with: `./hack/update-deps.sh`.
 1. Update your type definitions with: `./hack/update-codegen.sh`.
 1. [Add new CRD types](#adding-new-types)
+1. [Add and run tests](./test/README.md#tests)
 
 To make changes to these CRDs, you will probably interact with:
 

README.md

@@ -8,6 +8,14 @@ This repo contains the API definition of the Pipeline CRD and an on cluster impl
 The goal of the Pipeline CRD is to provide k8s-style resources that allow the
 declaration of CI/CD-style pipelines, which can be backed by any arbitrary implementation.
 
+## Status
+
+Right now the Pipeline CRD exists as [an API only](#features), and work is in progress to create
+an implementation of the API. Once [we hit our first milestone](https://github.com/knative/build-pipeline/milestone/1),
+you will be able to use this implementation to deploy and execute a simple Pipeline.
+
+## Features
+
 Features the Pipeline CRD will support include:
 
 * Conditional, parallel and distributed execution
@@ -44,7 +52,7 @@ High level details of this design:
 * [Tasks](#tasks) can depend on artifacts, output and parameters created by other tasks.
 * [Resources](#resources) are the artifacts used as inputs and outputs of TaskRuns.
 
-## Task
+### Task
 
 `Task` is a CRD that knows how to instantiate a [Knative Build](https://github.com/knative/build),
 either from a series of `steps` (i.e. [Builders](https://github.com/knative/docs/blob/master/build/builder-contract.md))
@@ -55,15 +63,15 @@ from is not known to a task, so they can be provided by a Pipeline or by a user
 `Tasks` are basically [Knative BuildTemplates](https://github.com/knative/build-templates)
 with additional input types and clearly defined outputs.
 
-## Pipeline
+### Pipeline
 
 `Pipeline` describes a graph of [Tasks](#task) to execute. It defines the DAG
 and expresses how all inputs (including [PipelineParams](#pipelineparams) and outputs
 from previous `Tasks`) feed into each `Task`.
 
 Dependencies between parameters or inputs/outputs are expressed as references to k8s objects.
 
-## PipelineParams
+### PipelineParams
 
 `PipelineParams` contains parameters for a [Pipeline](#pipeline). One `Pipeline`
 can be invoked with many different instances of `PipelineParams`, which can allow
@@ -73,7 +81,7 @@ for scenarios such as running against PRs and against a user’s personal setup.
 * Which **serviceAccount** to use (provided to all tasks)
 * Where **results** are stored (e.g. in GCS)
 
-## TaskRun
+### TaskRun
 
 Creating a `TaskRun` will invoke a [Task](#task), running all of the steps until completion
 or failure. Creating a `TaskRun` will require satisfying all of the input requirements of the
@@ -84,7 +92,7 @@ outputs, and in the future we may want to transition `Builds` to become `Tasks`.
 
 `TaskRuns` can be created directly by a user or by a [PipelineRun](#pipelinerun).
 
-### TaskRun Status
+#### TaskRun Status
 
 Once a `TaskRun` has been created, it will start excuting its steps
 sequentially. The `conditions` field will be updated as the `TaskRun`
@@ -99,7 +107,7 @@ executes:
 When the `TaskRun` has completed, the `steps` field will indicate
 the exit code of all steps that completed.
 
-## PipelineRun
+### PipelineRun
 
 Creating a `PipelineRun` executes the pipeline, creating [TaskRuns](#taskrun) for each task
 in the pipeline.
@@ -111,7 +119,7 @@ A `PipelineRun` could be created:
 * In response to an event (e.g. in response to a Github event, possibly processed via
   [Knative eventing](https://github.com/knative/eventing))
 
-### PipelineRun Status
+#### PipelineRun Status
 
 Once a `PipelineRun` has been created, it will start excuting the DAG
 of its Tasks by creating a [`TaskRun`](#taskrun) for each of them. The

test/README.md

@@ -0,0 +1,118 @@
+# Tests
+
+## Unit tests
+
+Unit tests live side by side with the code they are testing and can be run with:
+
+```shell
+go test ./...
+```
+
+_By default `go test` will not run [the integration tests](#integration-tests), which need
+`-tags=e2e` to be enabled._
+
+## Integration tests
+
+Integration tests live in this directory. To run these tests, you must provide `go` with
+`-tags=e2e`, and you may want to use some of the [common flags](#common-flags):
+
+```shell
+go test -v -count=1 -tags=e2e ./test
+```
+
+You can also use
+[all of flags defined in `knative/pkg/test`](https://github.com/knative/pkg/tree/master/test#flags).
+
+### Flags
+
+* By default the e2e tests against the current cluster in `~/.kube/config`
+  using the environment specified in [your environment variables](/DEVELOPMENT.md#environment-setup).
+* Since these tests are fairly slow, running them with logging
+  enabled is recommended (`-v`).
+* Using [`--logverbose`](#output-verbose-log) to see the verbose log output from test as well as from k8s libraries.
+* Using `-count=1` is [the idiomatic way to disable test caching](https://golang.org/doc/go1.10#test)
+
+You can [use test flags](#flags) to control the environment
+your tests run against, i.e. override [your environment variables](/DEVELOPMENT.md#environment-setup):
+
+```bash
+go test -v -tags=e2e -count=1 ./test --kubeconfig ~/special/kubeconfig --cluster myspecialcluster
+```
+
+Tests importing [`github.com/knative/build-pipline/test`](#adding-integration-tests) recognize these
+[all flags added by `knative/pkg/test`](https://github.com/knative/pkg/tree/master/test#flags).
+
+_Note the environment variable `K8S_CLUSTER_OVERRIDE`, while used by [knative/serving](https://github.com/knative/serving)
+and not by this project, will override the cluster used by the integration tests since they use
+[the same libs to get these flags](https://github.com/knative/serving)._
+
+### Adding integration tests
+
+In the [`test`](/test/) dir you will find several libraries in the `test` package
+you can use in your tests.
+
+This library exists partially in this directory and partially in
+[`knative/pkg/test`](https://github.com/knative/pkg/tree/master/test).
+
+The libs in this dir can:
+
+* [`init.go`](./init.go) initializes anything needed globally be the tests
+* [Get access to client objects](#get-access-to-client-objects)
+
+All integration tests _must_ be marked with the `e2e` [build constraint](https://golang.org/pkg/go/build/)
+so that `go test ./...` can be used to run only [the unit tests](#unit-tests), i.e.:
+
+```go
+// +build e2e
+```
+
+#### Get access to client objects
+
+To initialize client objects use [the command line flags](#use-flags)
+which describe the environment:
+
+```go
+func setup(t *testing.T) *test.Clients {
+    clients, err := test.NewClients(kubeconfig, cluster, namespaceName)
+    if err != nil {
+        t.Fatalf("Couldn't initialize clients: %v", err)
+    }
+    return clients
+}
+```
+
+The `Clients` struct contains initialized clients for accessing:
+
+* Kubernetes objects
+* [`Pipelines`](https://github.com/knative/build-pipeline#pipeline)
+* TODO: incrementally add clients for other types
+
+For example, to create a `Pipeline`:
+
+```bash
+_, err = clients.PipelineClient.Pipelines.Create(test.Route(namespaceName, pipelineName))
+```
+
+And you can use the client to clean up resources created by your test (e.g. in
+[your test cleanup](https://github.com/knative/pkg/tree/master/test#ensure-test-cleanup)):
+
+```go
+func tearDown(clients *test.Clients) {
+    if clients != nil {
+        clients.Delete([]string{routeName}, []string{configName})
+    }
+}
+```
+
+_See [clients.go](./clients.go)._
+
+## Presubmit tests
+
+[`presubmit-tests.sh`](./presubmit-tests.sh) is the entry point for all tests
+[run on presubmit by Prow](../CONTRIBUTING.md#pull-request-process).
+
+You can run this locally with:
+
+```shell
+test/presubmit-tests.sh
+```

test/clients.go

@@ -0,0 +1,54 @@
+/*
+Copyright 2018 Knative Authors LLC
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+    http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// This file contains an object which encapsulated k8s clients which are useful for integration tests.
+
+package test
+
+import (
+	"fmt"
+
+	"github.com/knative/build-pipeline/pkg/client/clientset/versioned"
+	"github.com/knative/build-pipeline/pkg/client/clientset/versioned/typed/pipeline/v1alpha1"
+	knativetest "github.com/knative/pkg/test"
+)
+
+// Clients holds instances of interfaces for making requests to the Pipeline controllers.
+type Clients struct {
+	KubeClient     *knativetest.KubeClient
+	PipelineClient v1alpha1.PipelineInterface
+}
+
+// NewClients instantiates and returns several clientsets required for making requests to the
+// Pipeline cluster specified by the combination of clusterName and configPath. Clients can
+// make requests within namespace.
+func NewClients(configPath string, clusterName string, namespace string) (*Clients, error) {
+	clients := &Clients{}
+	cfg, err := knativetest.BuildClientConfig(configPath, clusterName)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create configuration obj from %s for cluster %s: %s", configPath, clusterName, err)
+	}
+
+	clients.KubeClient, err = knativetest.NewKubeClient(configPath, clusterName)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create kubeclient from config file at %s: %s", configPath, err)
+	}
+
+	cs, err := versioned.NewForConfig(cfg)
+	if err != nil {
+		return nil, fmt.Errorf("failed to create pipeline clientset from config file at %s: %s", configPath, err)
+	}
+	clients.PipelineClient = cs.PipelineV1alpha1().Pipelines(namespace)
+
+	return clients, nil
+}

test/init_test.go

@@ -0,0 +1,42 @@
+/*
+Copyright 2018 Knative Authors LLC
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+    http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// This file contains initialization logic for the tests, such as special magical global state that needs to be initialized.
+
+package test
+
+import (
+	"flag"
+	"os"
+	"testing"
+
+	"github.com/knative/pkg/test"
+	"github.com/knative/pkg/test/logging"
+)
+
+func initializeLogsAndMetrics() {
+	flag.Parse()
+	flag.Set("alsologtostderr", "true")
+	logging.InitializeLogger(test.Flags.LogVerbose)
+
+	if test.Flags.EmitMetrics {
+		logging.InitializeMetricExporter()
+	}
+}
+
+// TestMain initializes anything global needed by the tests. Right now this is just log and metric
+// setup since the log and metric libs we're using use global state :(
+func TestMain(m *testing.M) {
+	initializeLogsAndMetrics()
+	os.Exit(m.Run())
+}