Skip to content

Commit

Permalink
feat: Support TCP routes traffic splitting for Istio VirtualService (#…
Browse files Browse the repository at this point in the history 
…1659)

Signed-off-by: Imran Ismail <imran.codely@gmail.com>
  • Loading branch information
@imranismail
imranismail authored Aug 5, 2022
1 parent 731d55b commit 25f40d2
Show file tree
Hide file tree
Showing 19 changed files with 1,746 additions and 544 deletions.
73 changes: 64 additions & 9 deletions docs/features/traffic-management/istio.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ different versions of the Rollout during an update.

## How it works

Traffic splitting is accomplished in Istio by adjusting traffic weights defined in an
Traffic splitting is accomplished in Istio by adjusting traffic weights defined in an
[Istio VirtualService](https://istio.io/latest/docs/reference/config/networking/virtual-service/).
When using Argo Rollouts with Istio, a user deploys a VirtualService containing at least one
[HTTP route](https://istio.io/latest/docs/reference/config/networking/virtual-service/#HTTPRoute) containing two
Expand Down Expand Up @@ -46,7 +46,7 @@ spec:
stableService: stable-svc # required
trafficRouting:
istio:
virtualService:
virtualService:
name: rollout-vsvc # required
routes:
- primary # optional if there is a single route in VirtualService, required otherwise
Expand All @@ -57,7 +57,7 @@ spec:
```
The VirtualService must contain an HTTP route with a name referenced in the Rollout, containing
two route destinations with `host` values that match the `canaryService` and `stableService`
two route destinations with `host` values that match the `canaryService` and `stableService`
referenced in the Rollout. If the VirtualService is defined in a different namespace than the rollout,
its name should be `rollout-vsvc.<vsvc namespace name>`. Note that Istio requires that all weights add to
100, so the initial weights can be 100% to stable, and 0% to canary.
Expand Down Expand Up @@ -127,7 +127,7 @@ During the lifecycle of a Rollout update, Argo Rollouts will continuously:

!!! note

Rollout does not make any other assumptions about the fields within the VirtualService or the Istio mesh. The user could specify additional configurations for the VirtualService like URI rewrite rules on the primary route or any other route if desired. The user can also create specific DestinationRules for each of the services.
Rollout does not make any other assumptions about the fields within the VirtualService or the Istio mesh. The user could specify additional configurations for the VirtualService like URI rewrite rules on the primary route or any other route if desired. The user can also create specific DestinationRules for each of the services.


## Subset-level Traffic Splitting
Expand Down Expand Up @@ -159,7 +159,7 @@ spec:
canary:
trafficRouting:
istio:
virtualService:
virtualService:
name: rollout-vsvc # required
routes:
- primary # optional if there is a single route in VirtualService, required otherwise
Expand Down Expand Up @@ -194,7 +194,7 @@ spec:
```

The VirtualService must contain an HTTP route with a name referenced in the Rollout, containing
two route destinations with `subset` values that match the `canarySubsetName` and `stableSubsetName`
two route destinations with `subset` values that match the `canarySubsetName` and `stableSubsetName`
referenced in the Rollout. Note that Istio requires that all weights add to 100, so the initial
weights can be 100% to stable, and 0% to canary.

Expand Down Expand Up @@ -245,6 +245,61 @@ During the lifecycle of a Rollout using Istio DestinationRule, Argo Rollouts wil
* modify the DestinationRule `spec.subsets[].labels` to contain the `rollouts-pod-template-hash`
label of the canary and stable ReplicaSets

## TCP Traffic Splitting

!!! important

Available since v1.2.2

Support for splitting TCP traffic was introduced and requires the Rollout to define the following fields:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: rollout-example
spec:
...
strategy:
canary:
canaryService: canary-svc # required
stableService: stable-svc # required
trafficRouting:
istio:
virtualService:
name: rollout-vsvc # required
tcpRoutes:
# Below fields are optional but if defined, they should match exactly with at least one of the TCP route match rules in your VirtualService
- port: 3000 # Only required if you want to match any rule in your VirtualService which contains this port
steps:
- setWeight: 5
- pause:
duration: 10m
```

The VirtualService must contain a TCP route with a matching port referenced in the Rollout

```yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: rollout-vsvc
spec:
gateways:
- istio-rollout-gateway
hosts:
- istio-rollout.dev.argoproj.io
tcp:
- match:
- port: 3000
route:
- destination:
host: stable-svc # referenced in canary.stableService
weight: 100
- destination:
host: canary-svc # referenced in canary.canaryService
weight: 0
```

## Multicluster Setup
If you have [Istio multicluster setup](https://istio.io/latest/docs/setup/install/multicluster/)
Expand Down Expand Up @@ -278,8 +333,8 @@ rules:
- update
- patch
```
Note: If Argo Rollout controller is also installed in the Istio primary cluster, then you can reuse the
`argo-rollouts-clusterrole` ClusterRole instead of creating a new one.
Note: If Argo Rollout controller is also installed in the Istio primary cluster, then you can reuse the
`argo-rollouts-clusterrole` ClusterRole instead of creating a new one.
3. Link the `ClusterRole` with the `ServiceAccount` in the Istio primary cluster.
```yaml
apiVersion: rbac.authorization.k8s.io/v1
Expand Down Expand Up @@ -420,7 +475,7 @@ other controllers (e.g. Argo Rollouts) controller manage them instead.

## Alternatives Considered

### Rollout ownership over the Virtual Service
### Rollout ownership over the Virtual Service

An early design alternative was that instead of the controller modifying a referenced VirtualService, the Rollout controller would create, manage, and own a Virtual Service. While this approach is GitOps friendly, it introduces other issues:

Expand Down
50 changes: 46 additions & 4 deletions docs/getting-started/istio/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,13 +57,15 @@ spec:
sniHosts:
- reviews.bookinfo.com
- localhost
...
tcpRoutes:
# Below fields are optional but if defined, they should match exactly with at least one of the TCP route match rules in your VirtualService
- port: 8020 # Only required if you want to match any rule in your VirtualService which contains this port
```
The VirtualService and route referenced in either `trafficRouting.istio.virtualService` or
`trafficRouting.istio.virtualServices`. `trafficRouting.istio.virtualServices` helps in adding
one or more virtualServices unlike `trafficRouting.istio.virtualService` where only single virtualService can be added.
This is required to have either HTTP or TLS, or both route specs that splits between the stable and the canary
This is required to have either HTTP, TLS, TCP or a mixed route specs that splits between the stable and the canary
services referenced in the rollout. If the route is HTTPS/TLS, we can match it based on the
given port number and/or SNI hosts. Note that both of them are optional and only needed if you
want to match any rule in your VirtualService which contains these.
Expand Down Expand Up @@ -113,6 +115,16 @@ spec:
- destination:
host: rollouts-demo-canary # Should match rollout.spec.strategy.canary.canaryService
weight: 0
tcp:
- match:
- port: 8020 # Should match the port number of the route defined in rollout.spec.strategy.canary.trafficRouting.istio.virtualServices.tcpRoutes
route:
- destination:
host: rollouts-demo-stable # Should match rollout.spec.strategy.canary.stableService
weight: 100
- destination:
host: rollouts-demo-canary # Should match rollout.spec.strategy.canary.canaryService
weight: 0
```

```yaml
Expand Down Expand Up @@ -150,6 +162,16 @@ spec:
- destination:
host: rollouts-demo-canary # Should match rollout.spec.strategy.canary.canaryService
weight: 0
tcp:
- match:
- port: 8020 # Should match the port number of the route defined in rollout.spec.strategy.canary.trafficRouting.istio.virtualServices.tcpRoutes
route:
- destination:
host: rollouts-demo-stable # Should match rollout.spec.strategy.canary.stableService
weight: 100
- destination:
host: rollouts-demo-canary # Should match rollout.spec.strategy.canary.canaryService
weight: 0
```

Run the following commands to deploy:
Expand Down Expand Up @@ -249,6 +271,16 @@ spec:
- destination:
host: rollouts-demo-canary
weight: 5
tcp:
- match:
- port: 8020
route:
- destination:
host: rollouts-demo-stable
weight: 95
- destination:
host: rollouts-demo-canary
weight: 5
```

```yaml
Expand Down Expand Up @@ -287,7 +319,17 @@ spec:
- destination:
host: rollouts-demo-canary
weight: 5
tcp:
- match:
- port: 8020
route:
- destination:
host: rollouts-demo-stable
weight: 95
- destination:
host: rollouts-demo-canary
weight: 5
```

As the Rollout progresses through steps, the HTTP and/or TLS route(s) destination weights will be
adjusted to match the current `setWeight` of the steps.
As the Rollout progresses through steps, the HTTP, TLS, and/or TCP route(s) destination weights will be
adjusted to match the current `setWeight` of the steps.
16 changes: 16 additions & 0 deletions manifests/crds/rollout-crd.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -760,6 +760,14 @@ spec:
items:
type: string
type: array
tcpRoutes:
items:
properties:
port:
format: int64
type: integer
type: object
type: array
tlsRoutes:
items:
properties:
Expand All @@ -784,6 +792,14 @@ spec:
items:
type: string
type: array
tcpRoutes:
items:
properties:
port:
format: int64
type: integer
type: object
type: array
tlsRoutes:
items:
properties:
Expand Down
16 changes: 16 additions & 0 deletions manifests/install.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11761,6 +11761,14 @@ spec:
items:
type: string
type: array
tcpRoutes:
items:
properties:
port:
format: int64
type: integer
type: object
type: array
tlsRoutes:
items:
properties:
Expand All @@ -11785,6 +11793,14 @@ spec:
items:
type: string
type: array
tcpRoutes:
items:
properties:
port:
format: int64
type: integer
type: object
type: array
tlsRoutes:
items:
properties:
Expand Down
16 changes: 16 additions & 0 deletions manifests/namespace-install.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11761,6 +11761,14 @@ spec:
items:
type: string
type: array
tcpRoutes:
items:
properties:
port:
format: int64
type: integer
type: object
type: array
tlsRoutes:
items:
properties:
Expand All @@ -11785,6 +11793,14 @@ spec:
items:
type: string
type: array
tcpRoutes:
items:
properties:
port:
format: int64
type: integer
type: object
type: array
tlsRoutes:
items:
properties:
Expand Down
18 changes: 18 additions & 0 deletions pkg/apiclient/rollout/rollout.swagger.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -995,6 +995,13 @@
"$ref": "#/definitions/github.com.argoproj.argo_rollouts.pkg.apis.rollouts.v1alpha1.TLSRoute"
},
"description": "A list of TLS/HTTPS routes within VirtualService to edit. If omitted, VirtualService must have a single route of this type."
},
"tcpRoutes": {
"type": "array",
"items": {
"$ref": "#/definitions/github.com.argoproj.argo_rollouts.pkg.apis.rollouts.v1alpha1.TCPRoute"
},
"description": "A list of TCP routes within VirtualService to edit. If omitted, VirtualService must have a single route of this type."
}
},
"title": "IstioVirtualService holds information on the virtual service the rollout needs to modify"
Expand Down Expand Up @@ -1689,6 +1696,17 @@
},
"title": "StringMatch Used to define what type of matching we will use exact, prefix, or regular expression"
},
"github.com.argoproj.argo_rollouts.pkg.apis.rollouts.v1alpha1.TCPRoute": {
"type": "object",
"properties": {
"port": {
"type": "string",
"format": "int64",
"description": "Port number of the TCP Route desired to be matched in the given Istio VirtualService."
}
},
"description": "TCPRoute holds the information on the virtual service's TCP routes that are desired to be matched for changing weights."
},
"github.com.argoproj.argo_rollouts.pkg.apis.rollouts.v1alpha1.TLSRoute": {
"type": "object",
"properties": {
Expand Down
1 change: 1 addition & 0 deletions pkg/apis/api-rules/violation_exceptions.list
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis
API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1,ExperimentStatus,TemplateStatuses
API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1,IstioTrafficRouting,VirtualServices
API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1,IstioVirtualService,Routes
API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1,IstioVirtualService,TCPRoutes
API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1,IstioVirtualService,TLSRoutes
API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1,KayentaMetric,Scopes
API rule violation: list_type_missing,github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1,MetricResult,Measurements
Expand Down
Loading

0 comments on commit 25f40d2

Please sign in to comment.