Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A/B testing support for Gloo Edge ingress controller #758

Closed
wants to merge 16 commits into from
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ For more details on how the canary analysis and promotion works please [read the
| Feature | Contour | Gloo | NGINX | Skipper | Traefik |
| ------------------------------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------ |
| Canary deployments (weighted traffic) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| A/B testing (headers and cookies routing) | :heavy_check_mark: | :heavy_minus_sign: | :heavy_check_mark: | :heavy_minus_sign: | :heavy_minus_sign: |
| A/B testing (headers and cookies routing) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_minus_sign: | :heavy_minus_sign: |
| Blue/Green deployments (traffic switch) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Webhooks (acceptance/load testing) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Manual gating (approve/pause/resume) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
Expand Down
15 changes: 13 additions & 2 deletions artifacts/flagger/account.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -153,8 +153,19 @@ rules:
resources:
- upstreams
- upstreams/finalizers
- upstreamgroups
- upstreamgroups/finalizers
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- gateway.solo.io
resources:
- routetables
- routetables/finalizers
verbs:
- get
- list
Expand Down
15 changes: 13 additions & 2 deletions charts/flagger/templates/rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,19 @@ rules:
resources:
- upstreams
- upstreams/finalizers
- upstreamgroups
- upstreamgroups/finalizers
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- gateway.solo.io
resources:
- routetables
- routetables/finalizers
verbs:
- get
- list
Expand Down
23 changes: 13 additions & 10 deletions docs/gitbook/tutorials/gloo-progressive-delivery.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
# Gloo Canary Deployments

This guide shows you how to use the [Gloo](https://gloo.solo.io/) ingress controller and Flagger to automate canary deployments.
This guide shows you how to use the [Gloo Edge](https://gloo.solo.io/) ingress controller and Flagger to automate canary deployments.

![Flagger Gloo Ingress Controller](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/diagrams/flagger-gloo-overview.png)

## Prerequisites

Flagger requires a Kubernetes cluster **v1.11** or newer and Gloo ingress **1.3.5** or newer.
This guide was written for Flagger version **1.5.0** or higher. Prior versions of Flagger used Gloo upstream groups to handle
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does flagger versioning work? According to semver this is a new feature, so I put 1.5.0 here; please let me know what to put if that's not accurate.

canaries, but newer versions of Flagger use Gloo route tables to handle canaries as well as A/B testing.

Flagger requires a Kubernetes cluster **v1.11** or newer and Gloo Edge ingress **1.6.0** or newer.

Install Gloo with Helm v3:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note, the guide will not work as written until 1.6.0 is released in two weeks, as helm install installs the latest 1.5 release by default.

I can provide the --version flag to the helm install command if we want to hard-code the guide to a 1.6 version (for now, a beta). Thoughts?


Expand All @@ -31,7 +34,7 @@ helm upgrade -i flagger flagger/flagger \
## Bootstrap

Flagger takes a Kubernetes deployment and optionally a horizontal pod autoscaler (HPA),
then creates a series of objects (Kubernetes deployments, ClusterIP services and Gloo upstream groups).
then creates a series of objects (Kubernetes deployments, ClusterIP services and Gloo route tables).
These objects expose the application outside the cluster and drive the canary analysis and promotion.

Create a test namespace:
Expand All @@ -52,7 +55,7 @@ Deploy the load testing service to generate traffic during the canary analysis:
kubectl -n test apply -k github.com/weaveworks/flagger//kustomize/tester
```

Create an virtual service definition that references an upstream group that will be generated by Flagger
Create a virtual service definition that references a route table that will be generated by Flagger
(replace `app.example.com` with your own domain):

```yaml
Expand All @@ -68,8 +71,8 @@ spec:
routes:
- matchers:
- prefix: /
routeAction:
upstreamGroup:
delegateAction:
ref:
name: podinfo
namespace: test
```
Expand Down Expand Up @@ -168,7 +171,7 @@ horizontalpodautoscaler.autoscaling/podinfo-primary
service/podinfo
service/podinfo-canary
service/podinfo-primary
upstreamgroups.gloo.solo.io/podinfo
routetables.gateway.solo.io/podinfo
```

When the bootstrap finishes Flagger will set the canary status to initialized:
Expand Down Expand Up @@ -252,13 +255,13 @@ podinfod=stefanprodan/podinfo:3.1.2
Generate HTTP 500 errors:

```bash
watch curl -H 'Host: app.example.com' http://gateway-proxy-v2.gloo-system/status/500
watch curl -H 'Host: app.example.com' http://gateway-proxy.gloo-system/status/500
```

Generate high latency:

```bash
watch curl -H 'Host: app.example.com' http://gateway-proxy-v2.gloo-system/delay/2
watch curl -H 'Host: app.example.com' http://gateway-proxy.gloo-system/delay/2
```

When the number of failed checks reaches the canary analysis threshold, the traffic is routed back to the primary,
Expand Down Expand Up @@ -353,7 +356,7 @@ podinfod=stefanprodan/podinfo:3.1.3
Generate 404s:

```bash
watch curl -H 'Host: app.example.com' http://gateway-proxy.gloo-system/status/400
watch curl -H 'Host: app.example.com' http://gateway-proxy.gloo-system/status/404
```

Watch Flagger logs:
Expand Down
6 changes: 3 additions & 3 deletions docs/gitbook/usage/deployment-strategies.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

Flagger can run automated application analysis, promotion and rollback for the following deployment strategies:
* **Canary Release** (progressive traffic shifting)
* Istio, Linkerd, App Mesh, NGINX, Skipper, Contour, Gloo, Traefik
* Istio, Linkerd, App Mesh, NGINX, Skipper, Contour, Gloo Edge, Traefik
* **A/B Testing** (HTTP headers and cookies traffic routing)
* Istio, App Mesh, NGINX, Contour
* Istio, App Mesh, NGINX, Contour, Gloo Edge
* **Blue/Green** (traffic switching)
* Kubernetes CNI, Istio, Linkerd, App Mesh, NGINX, Contour, Gloo
* Kubernetes CNI, Istio, Linkerd, App Mesh, NGINX, Contour, Gloo Edge
* **Blue/Green Mirroring** (traffic shadowing)
* Istio

Expand Down
15 changes: 13 additions & 2 deletions kustomize/base/flagger/rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -143,8 +143,19 @@ rules:
resources:
- upstreams
- upstreams/finalizers
- upstreamgroups
- upstreamgroups/finalizers
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
- apiGroups:
- gateway.solo.io
resources:
- routetables
- routetables/finalizers
verbs:
- get
- list
Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/gloo/register.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
package gloo

const (
GroupName = "gloo.solo.io"
GroupName = "gateway.solo.io"
)
2 changes: 1 addition & 1 deletion pkg/apis/gloo/v1/doc.go
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// +k8s:deepcopy-gen=package

// Package v1 is the v1 version of the API.
// +groupName=gloo.solo.io
// +groupName=gateway.solo.io
package v1
4 changes: 2 additions & 2 deletions pkg/apis/gloo/v1/register.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ var (
// Adds the list of known types to Scheme.
func addKnownTypes(scheme *runtime.Scheme) error {
scheme.AddKnownTypes(SchemeGroupVersion,
&UpstreamGroup{},
&UpstreamGroupList{},
&RouteTable{},
&RouteTableList{},
)
metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
return nil
Expand Down
47 changes: 40 additions & 7 deletions pkg/apis/gloo/v1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,48 @@ import (
// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// UpstreamGroup is a specification for a Gloo UpstreamGroup resource
type UpstreamGroup struct {
// RouteTable is a specification for a Gloo RouteTable resource
type RouteTable struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

Spec UpstreamGroupSpec `json:"spec"`
Spec RouteTableSpec `json:"spec"`
}

type UpstreamGroupSpec struct {
type RouteTableSpec struct {
Routes []Route `json:"routes,omitempty"`
}

type Route struct {
Matchers []Matcher `json:"matchers,omitempty"`
Action RouteAction `json:"routeAction,omitempty"`
InheritablePathMatchers bool `json:"inheritablePathMatchers,omitempty"`
}

type Matcher struct {
Headers []HeaderMatcher `json:"headers,omitempty"`
QueryParameterMatchers []QueryParameterMatcher `json:"queryParameters,omitempty"`
Methods []string `json:"methods,omitempty"`
}

type HeaderMatcher struct {
Name string `json:"name,omitempty"`
Value string `json:"value,omitempty"`
Regex bool `json:"regex,omitempty"`
InvertMatch bool `json:"invertMatch,omitempty"`
}

type QueryParameterMatcher struct {
Name string `json:"name,omitempty"`
Value string `json:"value,omitempty"`
Regex bool `json:"regex,omitempty"`
}

type RouteAction struct {
Destination MultiDestination `json:"multi,omitempty"`
}

type MultiDestination struct {
Destinations []WeightedDestination `json:"destinations,omitempty"`
}

Expand All @@ -40,10 +73,10 @@ type ResourceRef struct {

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// UpstreamGroupList is a list of UpstreamGroup resources
type UpstreamGroupList struct {
// RouteTableList is a list of RouteTable resources
type RouteTableList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata"`

Items []UpstreamGroup `json:"items"`
Items []RouteTable `json:"items"`
}
Loading