Support arbitrary traffic splitting

[deliahu]

Description

Support traffic splitting

API (draft)

- name: iris-classifier
  kind: TrafficSplitter
  apis:
    - name: iris-classifierA
      weight: 50
    - name: iris-classifierB
      weight: 50
  networking:
    endpoint: my-endpoint
    api_gateway: public

- name: iris-classifierA
  kind: SyncAPI
  predictor:
    type: python
    path: predictor.py
    config:
      model: s3://cortex-examples/pytorch/iris-classifier/weights.pth

- name: iris-classifierB
  kind: SyncAPI
  predictor:
    type: python
    path: predictor.py
    config:
      model: s3://cortex-examples/pytorch/iris-classifier/weights.pth

Virtual Service:

https://istio.io/latest/docs/reference/config/networking/virtual-service/#VirtualService

type Destination struct {
	ServiceName string
	ServicePort int32
	Weight      int32
}

type VirtualServiceSpec struct {
	Name         string
	Gateways     []string
	Destinations []Destination
	Path         string
	Rewrite      *string
	Labels       map[string]string
	Annotations  map[string]string
}

Validations

• When creating or updating a "splitter", validate that all APIs listed either exist already, or are going to be deployed in the same file
• When deleting an API, validate that it's not referenced by any existing "splitters"