copy changes to correct file

Signed-off-by: JENNIFER RONDEAU <jrondeau@heptio.com>
projectcontour · May 18, 2018 · 26eb8a4 · 26eb8a4
1 parent f843803
commit 26eb8a4
Showing 1 changed file with 188 additions and 67 deletions.
diff --git a/deployment/README.md b/deployment/README.md
@@ -1,110 +1,231 @@
-# Gimbal Discoverer
+# Deployment
+<!-- TOC -->
+
+- [Deployment](#deployment)
+    - [Prerequisites](#prerequisites)
+    - [Deploy Contour](#deploy-contour)
+    - [Deploy Discoverers](#deploy-discoverers)
+        - [Kubernetes](#kubernetes)
+        - [Openstack](#openstack)
+    - [Deploy Prometheus](#deploy-prometheus)
+        - [Stateful Deployment](#stateful-deployment)
+        - [Quick start](#quick-start)
+        - [Access the Prometheus web UI](#access-the-prometheus-web-ui)
+        - [Access the Alertmanager web UI](#access-the-alertmanager-web-ui)
+    - [Deploy Grafana](#deploy-grafana)
+        - [Quick start](#quick-start-1)
+        - [Access the Grafana UI](#access-the-grafana-ui)
+        - [Configure Grafana](#configure-grafana)
+            - [Configure datasource](#configure-datasource)
+            - [Dashboards](#dashboards)
+                - [Add Sample Kubernetes Dashboard](#add-sample-kubernetes-dashboard)
+    - [Validation](#validation)
+        - [Discovery cluster](#discovery-cluster)
+        - [Gimbal cluster](#gimbal-cluster)
+
+<!-- /TOC -->
+
+## Prerequisites
+
+- A copy of this repository. Download, or clone: 
+
+  ```sh
+  $ git clone git@github.com:heptio/gimbal.git
+  ```
+
+- A single Kubernetes cluster to deploy Gimbal
+- Kubernetes or Openstack clusters with flat networking. That is, each Pod has a routable IP address on the network.
+
+## Deploy Contour
+
+```sh
+# Navigate to deployment directory
+$ cd deployment
+
+# Deploy Contour
+$ kubectl create -f contour/
+```
 
-[![Build Status](https://travis-ci.com/heptio/gimbal.svg?token=dGsEGqM7L7s2vaK7wDXC&branch=master)](https://travis-ci.com/heptio/gimbal)
+The deployment includes sample Network Policies that restrict access to Contour and Envoy. The policies explicitly allow access from Prometheus to scrape for metrics. 
 
-## Overview
-The Gimbal Discoverer currently can monitor two systems, Kubernetes and Openstack. The Discoverers perform service discovery of remote clusters by finding remote endpoints and synchronizing them to a central Kubernetes cluster as Services & Endpoints. 
+**NOTE**: The current configuration exposes the `/stats` path from the Envoy Admin UI so that Prometheus can scrape for metrics.
 
-### Kubernetes 
-The Kubernetes Discoverer monitors available Services and Endpoints for a single Kubernetes cluster. The credentials to access each API server are provided with a Kubernetes Secret.
+For additional information about Contour, see [the Gimbal architecture doc](../docs/gimbal-architecture.md).
 
-The Discoverer leverages the watch operation of the Kubernetes API to receive changes dynamically, instead of polling the API. All available Services and Endpoints are synchronized to the Team namespace that matches the source system.
+## Deploy Discoverers
 
-### Openstack
-The Openstack Discoverer monitors all configured Load Balancers as a Service (LBaaS) plus their corresponding Members. They are synchronized to the Team namespace as Services and Endpoints. The namespace ia configured as the OpenStack TenantName. 
+Service discovery is enabled with Discoverers, which have both Kubernetes and Openstack implementations.
 
-The Discoverer polls the Openstack API on a customizable interval.  
+```sh
+# Create gimbal-discoverer namespace
+kubectl create -f gimbal-discoverer/01-common.yaml
+```
 
-## Get started
+### Kubernetes
 
-#### Args
+The Kubernetes Discoverer is responsible for looking at all services and endpoints in a Kubernetes cluster and synchronizing them to the host cluster. 
 
-The following arguments are available to customize the Discoverer:
+[Credentials](../docs/kubernetes-discoverer.md#credentials) to the remote cluster must be created as a Secret.
 
-| flag  | default  | description  | discoverer | 
-|---|---|---|---|
-| --version  |  false | Show version and quit  | ALL | 
-| --num-threads  | 2  |  Specify number of threads to use when processing queue items. | ALL
-| --gimbal-kubecfg-file  | ""  | Location of kubecfg file for access to kubernetes cluster hosting Gimbal | ALL
-| --discover-kubecfg-file | ""  | Location of kubecfg file for access to remote kubernetes cluster to watch for services / endpoints | Kubernetes
-| --cluster-name  | ""  |   Name of cluster scraping for services & endpoints | ALL
-| --debug | false | Enable debug logging | ALL
-| --reconciliation-period | 30s | The interval of time between reconciliation loop runs | Openstack
-| --http-client-timeout | 5s | The HTTP client request timeout | Openstack
-| --openstack-certificate-authority | "" | Path to cert file of the OpenStack API certificate authority | Openstack
-| --resync-interval | 30m | Resync period for Kubernetes watch client | 
+```sh
+# Kubernetes secret
+$ kubectl -n gimbal-discovery create secret generic remote-discover-kubecfg \
+    --from-file=config=./config \
+    --from-literal=cluster-name=node02
 
-## Deployment
+# Deploy Discoverer
+$ kubectl apply -f gimbal-discoverer/02-kubernetes-discoverer.yaml
+```
 
-The discoverer can be deployed by utilizing the included deployment files. They contain the correct RBAC rules, as well as the deployment of the discoverer itself.
+For more information, see [the Kubernetes Discoverer doc](../docs/kubernetes-discoverer.md).
 
-_NOTE: Best practice would be to to create a service account user in the remote cluster who only has permissions to `watch`, `list` and `get` services & endpoints._
+### Openstack
 
-### Kubernetes
+The Openstack Discoverer is responsible for looking at all LBaaS and members in an Openstack cluster and synchronizing them to the host cluster. 
+
+[Credentials](../docs/openstack-discoverer.md#credentials) to the remote cluster must be created as a secret.
+
+```sh
+# Openstack secret
+$ kubectl -n gimbal-discovery create secret generic remote-discover-openstack \
+    --from-file=certificate-authority-data=./ca.pem \
+    --from-literal=cluster-name=openstack \
+    --from-literal=username=admin \
+    --from-literal=password=abc123 \
+    --from-literal=auth-url=https://api.openstack:5000/ \
+    --from-literal=tenant-name=heptio
+
+# Deploy Discoverer
+$ kubectl apply -f gimbal-discoverer/02-openstack-discoverer.yaml
 ```
-# Create namespace / deployment / rbac rules:
-$ kubectl apply -f deployment/kubernetes-discoverer
 
-# Create secret for remote k8s cluster:
-$ kubectl create secret generic remote-discover-kubecfg --from-file=./config -n gimbal-discovery
-```
+For more information, see [the OpenStack Discoverer doc](../docs/openstack-discoverer.md).
 
-### Openstack
-```
-# Create namespace / deployment / rbac rules:
-$ kubectl apply -f deployment/openstack-discoverer
+## Deploy Prometheus
 
-# Create secret for remote openstack cluster:
-$ kubectl create secret generic remote-discover-openstack --from-literal=keystoneUrl=http://openstack001:5000/v3/ --from-literal=neutronUrl=http://openstack001:9696/ --from-literal=username=someUser --from-literal=password=secretPassword --from-literal=userDomain=default --from-file=./cert.pem -n gimbal-discovery
-```
+A sample deployment of Prometheus and Alertmanager is provided that uses temporary storage. This deployment can be used for testing and development, but might not be suitable for all environments.
 
-## Development
+### Stateful Deployment
 
-### Kubernetes
+ A stateful deployment of Prometheus should use persistent storage with [Persistent Volumes and Persistent Volume Claims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) to maintain a correlation between a data volume and the Prometheus Pod. Persistent volumes can be static or dynamic and depends on the backend storage implementation utilized in environment in which the cluster is deployed. For more information, see the [Kubernetes documentation on types of persistent volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#types-of-persistent-volumes).
 
-The Kubernetes discoverer requires two configs, first is the Gimbal system which will run Contour and store services & endpoints, the other is the remote cluster to scrape for services & endpoints. The config file is standard kubeconfig file, just make sure it's named `config`. Please include any certs required to access to the remote cluster api:
+### Quick start
 
+```sh
+# Deploy 
+$ kubectl apply -f prometheus
+
+# Deploy kube-state-metrics to gather cluster information
+$ git clone https://github.com/kubernetes/kube-state-metrics.git
+$ cd kube-state-metrics
+$ kubectl apply -f kubernetes/
 ```
-$ go run cmd/kubernetes-discoverer/main.go --gimbal-kubecfg-file=./config --discover-kubecfg-file=./config --cluster-name=clustername
+
+### Access the Prometheus web UI
+
+```sh
+$ kubectl -n gimbal-monitoring port-forward $(kubectl -n gimbal-monitoring get pods -l app=prometheus -l component=server -o jsonpath='{.items[0].metadata.name}') 9090:9090
 ```
 
-### Openstack
+then go to [http://localhost:9090](http://localhost:9090) in your browser.
 
-The Openstack discoverer requires the config for the Gimbal Kubernetes cluster which will run Contour and store services & endpoints, the other is the remote cluster to scrape for load balancers and members. The config file is standard kubeconfig file, just make sure it's named `config`. Please include any certs required to access to the remote cluster api:
+### Access the Alertmanager web UI
 
+```sh
+$ kubectl -n gimbal-monitoring port-forward $(kubectl -n gimbal-monitoring get pods -l app=prometheus -l component=alertmanager -o jsonpath='{.items[0].metadata.name}') 9093:9093
 ```
-$ OS_USERNAME=user OS_PASSWORD=password OS_AUTH_URL=https://url OS_TENANT_NAME=tenant go run cmd/openstack-discoverer/main.go --gimbal-kubecfg-file=./config --cluster-name=clustername
+
+then go to [http://localhost:9093](http://localhost:9093) in your browser.
+
+## Deploy Grafana
+
+A sample deployment of Grafana is provided that uses temporary storage.
+
+### Quick start
+
+```sh
+# Deploy
+$ kubectl apply -f grafana/
+
+# Create secret with grafana credentials
+$ kubectl create secret generic grafana -n gimbal-monitoring \
+    --from-literal=grafana-admin-password=admin \
+    --from-literal=grafana-admin-user=admin 
 ```
 
-## Build / Test
+### Access the Grafana UI
 
+```sh
+$ kubectl port-forward $(kubectl get pods -l app=grafana -n gimbal-monitoring -o jsonpath='{.items[0].metadata.name}') 3000 -n gimbal-monitoring
 ```
-Create a binary:
-$ make build
 
-Run tests:
-$ make test
+then go to [http://localhost:3000](http://localhost:3000) in your browser. The username is `admin`.
+
+### Configure Grafana
+
+Grafana requires some configuration after it's deployed.
+
+#### Configure datasource
+
+1. On the main Grafana page, click **Add Datasource**
+2. For **Name** enter _prometheus_
+3. In the **Type** selector, choose _Prometheus_ 
+4. For the URL, enter `http://prometheus:9090`
+5. Click **Save & Test**
+6. Look for the message box _Data source is working_
+
+#### Configure dashboards
 
-Create container:
-$ REGISTRY=heptio make container
+Dashboards for Envoy and the Discovery components are included as part of the sample Grafana deployment.
 
-Push container: 
-$ REGISTRY=heptio make push
+##### Add Sample Kubernetes Dashboard
+
+Add a sample dashboard to validate that the data source is collecting data:
+
+1. On the main page, click the plus icon and choose **Import dashboard**
+2. Enter _1621_ in the first box
+3. In the **prometheus** section, choose the datasource that you just created
+4. Click **Import**
+
+## Validation
+
+### Discovery cluster
+
+This example deploys a sample application in the default namespace of [the discovered Kubernetes cluster that you created](#kubernetes).
+
+```sh
+# Deploy sample apps
+$ kubectl apply -f example-workload/deployment.yaml
 ```
 
-_NOTE: The registry ENV variable allow you to override the registry so custom images can be tested._
+### Gimbal cluster
 
-## Contributing
+Run the following commands on the Gimbal cluster to verify its components:
 
-Thanks for taking the time to join our community and start contributing!
+```sh
+# Verify Discoverer Components
+$ kubectl get po -n gimbal-discovery
+NAME                                         READY     STATUS    RESTARTS   AGE
+k8s-kubernetes-discoverer-55899dcb66-lgvnk   1/1       Running   0          5m
 
-#### Before you start
+# Verify Contour
+$ kubectl get po -n gimbal-contour
+NAME            READY     STATUS    RESTARTS   AGE
+contour-lq6mm   2/2       Running   0          5h
 
-* Please familiarize yourself with the [Code of
-Conduct](https://github.com/heptio/gimbal/blob/master/CODE_OF_CONDUCT.md) before contributing.
-* See [CONTRIBUTING.md](https://github.com/heptio/gimbal/blob/master/CONTRIBUTING.md) for instructions on the
-developer certificate of origin that we require.
+# Verify discovered services
+$ kubectl get svc -l gimbal.heptio.com/backend=node02 
+NAME                TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
+nginx-node02        ClusterIP   None         <none>        80/TCP    17m
+kuard-node02        ClusterIP   None         <none>        80/TCP    17m
 
-#### Pull requests
+# Deploy an Ingress route
+$ kubectl apply -f example-workload/ingressroute.yaml
 
-* We welcome pull requests. Feel free to dig through the [issues](10) and jump in.
+# Port forward to the Contour pod
+$ kubectl port-forward $(kubectl get pods -n gimbal-contour -l app=contour -o jsonpath='{.items[0].metadata.name}') 9000:80 -n gimbal-contour
+
+# Make a request to Gimbal cluster which will proxy traffic to the secondary cluster
+# The curl should respond with the kuard and nginx alternating between requests
+$ curl -i -H "Host: mixed.local" localhost:9000
+```