Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Istio traffic policy #200

Merged
merged 7 commits into from
Jun 12, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 22 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
[![release](https://img.shields.io/github/release/weaveworks/flagger/all.svg)](https://github.com/weaveworks/flagger/releases)

Flagger is a Kubernetes operator that automates the promotion of canary deployments
using Istio, App Mesh or NGINX routing for traffic shifting and Prometheus metrics for canary analysis.
using Istio, App Mesh, NGINX or Gloo routing for traffic shifting and Prometheus metrics for canary analysis.
The canary analysis can be extended with webhooks for running acceptance tests,
load tests or any other custom validation.

Expand Down Expand Up @@ -149,16 +149,27 @@ For more details on how the canary analysis and promotion works please [read the

## Features

| Feature | Istio | App Mesh | SuperGloo | NGINX Ingress |
| -------------------------------------------- | ------------------ | ------------------ |------------------ |------------------ |
| Canary deployments (weighted traffic) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| A/B testing (headers and cookies filters) | :heavy_check_mark: | :heavy_minus_sign: | :heavy_minus_sign: | :heavy_check_mark: |
| Load testing | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Webhooks (custom acceptance tests) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Request success rate check (L7 metric) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Request duration check (L7 metric) | :heavy_check_mark: | :heavy_minus_sign: | :heavy_check_mark: | :heavy_check_mark: |
| Custom promql checks | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Ingress gateway (CORS, retries and timeouts) | :heavy_check_mark: | :heavy_minus_sign: | :heavy_check_mark: | :heavy_check_mark: |
| Service Mesh Feature | Istio | App Mesh | SuperGloo |
| -------------------------------------------- | ------------------ | ------------------ |------------------ |
| Canary deployments (weighted traffic) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| A/B testing (headers and cookies filters) | :heavy_check_mark: | :heavy_minus_sign: | :heavy_minus_sign: |
| Load testing | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Webhooks (acceptance testing) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Request success rate check (L7 metric) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Request duration check (L7 metric) | :heavy_check_mark: | :heavy_minus_sign: | :heavy_check_mark: |
| Custom promql checks | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
| Ingress gateway (CORS, retries and timeouts) | :heavy_check_mark: | :heavy_minus_sign: | :heavy_check_mark: |

| Ingress Controller Feature | NGINX | Gloo |
| -------------------------------------------- | ------------------ | ------------------ |
| Canary deployments (weighted traffic) | :heavy_check_mark: | :heavy_check_mark: |
| A/B testing (headers and cookies filters) | :heavy_check_mark: | :heavy_minus_sign: |
| Load testing | :heavy_check_mark: | :heavy_check_mark: |
| Webhooks (acceptance testing) | :heavy_check_mark: | :heavy_check_mark: |
| Request success rate check (L7 metric) | :heavy_minus_sign: | :heavy_check_mark: |
| Request duration check (L7 metric) | :heavy_minus_sign: | :heavy_check_mark: |
| Custom promql checks | :heavy_check_mark: | :heavy_check_mark: |


## Roadmap

Expand Down
15 changes: 7 additions & 8 deletions artifacts/canaries/canary.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,19 @@ spec:
# Istio virtual service host names (optional)
hosts:
- app.istio.weavedx.com
# Istio traffic policy (optional)
trafficPolicy:
loadBalancer:
simple: LEAST_CONN
# HTTP match conditions (optional)
match:
- uri:
prefix: /
# HTTP rewrite (optional)
rewrite:
uri: /
# Envoy timeout and retry policy (optional)
headers:
request:
add:
x-envoy-upstream-rq-timeout-ms: "15000"
x-envoy-max-retries: "10"
x-envoy-retry-on: "gateway-error,connect-failure,refused-stream"
# HTTP timeout (optional)
timeout: 30s
# promote the canary without analysing it (default false)
skipAnalysis: false
canaryAnalysis:
Expand All @@ -54,7 +53,7 @@ spec:
# canary increment step
# percentage (0-100)
stepWeight: 5
# Istio Prometheus checks
# Prometheus checks
metrics:
- name: request-success-rate
# minimum req success rate (non 5xx responses)
Expand Down
2 changes: 2 additions & 0 deletions artifacts/flagger/account.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ rules:
resources:
- virtualservices
- virtualservices/status
- destinationrules
- destinationrules/status
verbs: ["*"]
- apiGroups:
- appmesh.k8s.aws
Expand Down
13 changes: 13 additions & 0 deletions artifacts/workloads/canary-destination.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: podinfo-canary
namespace: test
spec:
host: podinfo-canary
trafficPolicy:
loadBalancer:
consistentHash:
httpCookie:
name: user
ttl: 0s
2 changes: 1 addition & 1 deletion artifacts/workloads/canary-hpa.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ spec:
kind: Deployment
name: podinfo
minReplicas: 2
maxReplicas: 3
maxReplicas: 2
metrics:
- type: Resource
resource:
Expand Down
13 changes: 13 additions & 0 deletions artifacts/workloads/primary-destination.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: podinfo-primary
namespace: test
spec:
host: podinfo-primary
trafficPolicy:
loadBalancer:
consistentHash:
httpCookie:
name: user
ttl: 0s
2 changes: 1 addition & 1 deletion artifacts/workloads/primary-hpa.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ spec:
kind: Deployment
name: podinfo-primary
minReplicas: 2
maxReplicas: 4
maxReplicas: 2
metrics:
- type: Resource
resource:
Expand Down
19 changes: 6 additions & 13 deletions artifacts/workloads/virtual-service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,17 @@ metadata:
app: podinfo
spec:
gateways:
- public-gateway.istio-system.svc.cluster.local
- mesh
- public-gateway.istio-system.svc.cluster.local
hosts:
- podinfo.istio.weavedx.com
- podinfo
- app.istio.weavedx.com
http:
- route:
- destination:
host: podinfo-primary
port:
number: 9898
weight: 100
weight: 50
- destination:
host: podinfo
port:
number: 9898
weight: 0
timeout: 10s
retries:
attempts: 3
perTryTimeout: 2s
host: podinfo-canary
weight: 50
timeout: 5s
2 changes: 2 additions & 0 deletions charts/flagger/templates/rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ rules:
resources:
- virtualservices
- virtualservices/status
- destinationrules
- destinationrules/status
verbs: ["*"]
- apiGroups:
- appmesh.k8s.aws
Expand Down
44 changes: 35 additions & 9 deletions docs/gitbook/how-it-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,9 @@ the Istio Virtual Service. The container port from the target deployment should

### Istio routing

Flagger creates an Istio Virtual Service based on the Canary service spec. The service configuration lets you expose
an app inside or outside the mesh.
You can also define HTTP match conditions, URI rewrite rules, CORS policies, timeout and retries.
Flagger creates an Istio Virtual Service and Destination Rules based on the Canary service spec.
The service configuration lets you expose an app inside or outside the mesh.
You can also define traffic policies, HTTP match conditions, URI rewrite rules, CORS policies, timeout and retries.

The following spec exposes the `frontend` workload inside the mesh on `frontend.test.svc.cluster.local:9898`
and outside the mesh on `frontend.example.com`. You'll have to specify an Istio ingress gateway for external hosts.
Expand All @@ -129,6 +129,10 @@ spec:
# Istio virtual service host names (optional)
hosts:
- frontend.example.com
# Istio traffic policy (optional)
trafficPolicy:
loadBalancer:
simple: LEAST_CONN
# HTTP match conditions (optional)
match:
- uri:
Expand Down Expand Up @@ -198,18 +202,40 @@ spec:
route:
- destination:
host: podinfo-primary
port:
number: 9898
weight: 100
- destination:
host: podinfo-canary
port:
number: 9898
weight: 0
```

Flagger keeps in sync the virtual service with the canary service spec. Any direct modification to the virtual
service spec will be overwritten.
For each destination in the virtual service a rule is generated:

```yaml
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: frontend-primary
namespace: test
spec:
host: frontend-primary
trafficPolicy:
loadBalancer:
simple: LEAST_CONN
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: frontend-canary
namespace: test
spec:
host: frontend-canary
trafficPolicy:
loadBalancer:
simple: LEAST_CONN
```

Flagger keeps in sync the virtual service and destination rules with the canary service spec.
Any direct modification to the virtual service spec will be overwritten.

To expose a workload inside the mesh on `http://backend.test.svc.cluster.local:9898`,
the service spec can contain only the container port:
Expand Down
2 changes: 2 additions & 0 deletions docs/gitbook/usage/ab-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@ horizontalpodautoscaler.autoscaling/abtest-primary
service/abtest
service/abtest-canary
service/abtest-primary
destinationrule.networking.istio.io/abtest-canary
destinationrule.networking.istio.io/abtest-primary
virtualservice.networking.istio.io/abtest
```

Expand Down
2 changes: 2 additions & 0 deletions docs/gitbook/usage/progressive-delivery.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ horizontalpodautoscaler.autoscaling/podinfo-primary
service/podinfo
service/podinfo-canary
service/podinfo-primary
destinationrule.networking.istio.io/podinfo-canary
destinationrule.networking.istio.io/podinfo-primary
virtualservice.networking.istio.io/podinfo
```

Expand Down
23 changes: 12 additions & 11 deletions pkg/apis/flagger/v1alpha3/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,17 +115,18 @@ type CanaryStatus struct {
// CanaryService is used to create ClusterIP services
// and Istio Virtual Service
type CanaryService struct {
Port int32 `json:"port"`
PortName string `json:"portName,omitempty"`
Match []istiov1alpha3.HTTPMatchRequest `json:"match,omitempty"`
Rewrite *istiov1alpha3.HTTPRewrite `json:"rewrite,omitempty"`
Timeout string `json:"timeout,omitempty"`
Retries *istiov1alpha3.HTTPRetry `json:"retries,omitempty"`
Headers *istiov1alpha3.Headers `json:"headers,omitempty"`
CorsPolicy *istiov1alpha3.CorsPolicy `json:"corsPolicy,omitempty"`
//Istio
Gateways []string `json:"gateways,omitempty"`
Hosts []string `json:"hosts,omitempty"`
Port int32 `json:"port"`
PortName string `json:"portName,omitempty"`
Timeout string `json:"timeout,omitempty"`
// Istio
Gateways []string `json:"gateways,omitempty"`
Hosts []string `json:"hosts,omitempty"`
TrafficPolicy *istiov1alpha3.TrafficPolicy `json:"trafficPolicy,omitempty"`
Match []istiov1alpha3.HTTPMatchRequest `json:"match,omitempty"`
Rewrite *istiov1alpha3.HTTPRewrite `json:"rewrite,omitempty"`
Retries *istiov1alpha3.HTTPRetry `json:"retries,omitempty"`
Headers *istiov1alpha3.Headers `json:"headers,omitempty"`
CorsPolicy *istiov1alpha3.CorsPolicy `json:"corsPolicy,omitempty"`
// App Mesh
MeshName string `json:"meshName,omitempty"`
Backends []string `json:"backends,omitempty"`
Expand Down
25 changes: 15 additions & 10 deletions pkg/apis/flagger/v1alpha3/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading