Skip to content

Commit

Permalink
Merge pull request #3163 from cakuros/cakuros@github.com/ambassador-docs
Browse files Browse the repository at this point in the history 
Documentation updates for the Ambassador Install process and to introduce CRD based config.
  • Loading branch information
@ukclivecox
ukclivecox authored May 6, 2021
2 parents b8aa83d + e146cc3 commit 8249af5
Show file tree
Hide file tree
Showing 3 changed files with 124 additions and 27 deletions.
72 changes: 51 additions & 21 deletions doc/source/ingress/ambassador.md
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,30 @@
# Ingress with Ambassador

Seldon Core works well with [Ambassador](https://www.getambassador.io/), allowing a single ingress to be used to expose ambassador and [running machine learning deployments can then be dynamically exposed](https://kubernetes.io/blog/2018/06/07/dynamic-ingress-in-kubernetes/) through seldon-created ambassador configurations. In this doc we will discuss how your Seldon Deployments are exposed via Ambassador and how you can use both to do various production rollout strategies.
Seldon Core works well with [Ambassador](https://www.getambassador.io/), allowing a single ingress to be used to expose Ambassador and [running machine learning deployments that can then be dynamically exposed](https://kubernetes.io/blog/2018/06/07/dynamic-ingress-in-kubernetes/) through Seldon-created Ambassador configurations. Ambassador is a Kubernetes-native API Gateway built on Envoy Proxy. Managed entirely via Kubernetes Custom Resource Definitions, Ambassador provides powerful capabilities for traffic management, authentication, and observability. Ambassador has native integrations for popular service meshes, including Consul, Istio, and Linkerd. In this doc we will discuss how your Seldon Deployments are exposed via Ambassador and how you can use both to do various production rollout strategies.

## Installing Ambassador

You have two options when installing Ambassador:
You have [two options](https://www.getambassador.io/editions/) when installing Ambassador:

### Option 1: Ambassador API Gateway
### Option 1: Ambassador Edge Stack

The [Ambassador API Gateway](https://www.getambassador.io/products/api-gateway/) is open source and provides all the functionality of a traditional ingress controller.
The [Ambassador Edge Stack](https://www.getambassador.io/products/edge-stack/) is the easiest way to get started with Ambassador, providing easy, [Automatic TLS configuration](https://www.getambassador.io/docs/edge-stack/latest/topics/running/host-crd/#acme-support) in addition to other features, like [Authentication](https://www.getambassador.io/docs/edge-stack/latest/topics/using/filters/), [Rate Limiting](https://www.getambassador.io/docs/edge-stack/latest/topics/using/rate-limits/), and advanced routing behaviors, like [custom prefixes](https://www.getambassador.io/docs/edge-stack/latest/topics/using/intro-mappings/), [virtual hosting](https://www.getambassador.io/docs/edge-stack/latest/topics/using/headers/host/), [method](https://www.getambassador.io/docs/edge-stack/latest/topics/using/method/)/[query-parameter](https://www.getambassador.io/docs/edge-stack/latest/topics/using/query_parameters/) based routing, and many others.
Follow the [AES installation instructions](https://www.getambassador.io/docs/edge-stack/latest/tutorials/getting-started/) to install it on your Kubernetes cluster.

Using `helm` the steps can be summarised as
```bash
kubectl create namespace ambassador || echo "namespace ambassador exists"

helm repo add datawire https://www.getambassador.io
helm install ambassador datawire/ambassador \
--set image.repository=docker.io/datawire/ambassador \
--set crds.keep=false \
--namespace ambassador
```

### Option 2: Ambassador API Gateway (now Emissary Ingress)

The [Ambassador API Gateway](https://www.getambassador.io/products/api-gateway/) (now Emissary Ingress) is the fully open source version of Ambassador Edge Stack and provides all the functionality of a traditional ingress controller.
Follow the [Ambassador OSS instructions](https://www.getambassador.io/docs/latest/topics/install/install-ambassador-oss/) to install it on your kubernetes cluster.

Using `helm` the steps can be summarised as
Expand All @@ -17,26 +33,22 @@ kubectl create namespace ambassador || echo "namespace ambassador exists"

helm repo add datawire https://www.getambassador.io
helm install ambassador datawire/ambassador \
--set image.repository=quay.io/datawire/ambassador \
--set image.repository=docker.io/datawire/ambassador \
--set enableAES=false \
--set crds.keep=false \
--namespace ambassador
```

### Option 2: Ambassador Edge Stack

The [Ambassador Edge Stack](https://www.getambassador.io/products/edge-stack/) is the easiest way to get started with ambassador.
The `edgectl install` command will provision a load balancer, configure TLS, and provide you with an edgestack.me subdomain.
The edgestack.me subdomain allows the Ambassador Edge Stack to automatically provision TLS and HTTPS for a domain name.
To install AES using `edgectl` follow the [AES installation instructions](https://www.getambassador.io/docs/latest/topics/install/).
### TLS

Once the installation has finished, you can run `edgectl login --namespace=ambassador <ambassadorEndpoint>` to access the Ambassador Edge Policy Console where you can manage your deployment.
The 'Hosts' tab will provide information about the domain that Ambassador set up during the installation process.
It is highly recommended to utilize TLS to encrypt traffic that is coming into Ambassador. The default Ambassador Edge Stack installation comes with a self-signed certificate that will be used absent any other TLS configuration. Follow the [instructions](https://www.getambassador.io/docs/edge-stack/latest/howtos/tls-termination/) for setting up TLS on your cluster.

## Ambassador REST

Assuming Ambassador is exposed at `<ambassadorEndpoint>` and with a Seldon deployment name `<deploymentName>` running in a namespace `namespace`:

Note, if you chose to install the Ambassador Edge Stack then you will need to use https. You can either [set up TLS](https://www.getambassador.io/docs/edge-stack/latest/howtos/tls-termination/) or pass the `-k` flag in `curl` to allow the self-signed certificate.

For Seldon Core restricted to a namespace, `singleNamespace=true`, the endpoints exposed are:

* `http(s)://<ambassadorEndpoint>/seldon/<deploymentName>/api/v1.0/predictions`
Expand All @@ -46,9 +58,6 @@ For Seldon Core running cluster wide, `singleNamespace=false`, the endpoints exp

* `http(s)://<ambassadorEndpoint>/seldon/<namespace>/<deploymentName>/api/v1.0/predictions`

Note here that if you chose to install the Ambassador Edge Stack then you will need to use https and the `<AmbassadorEndpoint>` referenced above will be the domain name that Ambassador created for you (e.g. `random-name-1234.edgestack.me`)


## Example Curl

### Ambassador REST
Expand All @@ -59,10 +68,16 @@ If you installed the OSS Ambassador API Gateway, and assuming a Seldon Deploymen
curl -v 0.0.0.0:8003/seldon/mymodel/api/v1.0/predictions -d '{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}' -H "Content-Type: application/json"
```

Alternatively, if you installed the Ambassador Edge Stack, and assuming a Seldon Deployment `mymodel` with the Ambassador hostname `random-hostname-1234.edgestack.me`:
Alternatively, if you installed the Ambassador Edge Stack with TLS configured, and assuming a Seldon Deployment `mymodel` with the Ambassador hostname `example-hostname.com`:

```bash
curl -v https://random-hostname-1234.edgestack.me/seldon/mymodel/api/v1.0/predictions -d '{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}' -H "Content-Type: application/json"
curl -v https://example-hostname.com/seldon/mymodel/api/v1.0/predictions -d '{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}' -H "Content-Type: application/json"
```

If you did not, you can use the exposed IP address in place of `example-hostname` and pass the -k flag for insecure TLS.

```bash
curl -vk https://0.0.0.0/seldon/mymodel/api/v1.0/predictions -d '{"data":{"names":["a","b"],"tensor":{"shape":[2,2],"values":[0,0,1,1]}}}' -H "Content-Type: application/json"
```

## Ambassador Configuration Annotations Reference
Expand All @@ -86,7 +101,6 @@ All annotations should be placed in `spec.annotations`.

See below for details.


### Canary Deployments

Canary rollouts are available where you wish to push a certain percentage of traffic to a new model to test whether it works ok in production. To add a canary to your SeldonDeployment simply add a new predictor section and set the traffic levels for the main and canary to desired levels. For example:
Expand Down Expand Up @@ -162,6 +176,7 @@ A worked example for [header based routing](../examples/ambassador_headers.html)
To understand more about the Ambassador configuration for this see [their docs on header based routing](https://www.getambassador.io/reference/headers).

### Circuit Breakers

By preventing additional connections or requests to an overloaded Seldon Deployment, circuit breakers help improve resilience of your system.

You simply need to add some annotations to your Seldon Deployment resource.
Expand Down Expand Up @@ -204,9 +219,24 @@ See [AMBASSADOR_ID](https://www.getambassador.io/docs/latest/topics/running/runn

### Custom Amabassador configuration

The above discussed configurations should cover most cases but there maybe a case where you want to have a very particular Ambassador configuration under your control. You can acheieve this by adding your confguration as an annotation to your Seldon Deployment resource.
The above discussed configurations should cover most cases, however custom configuration is necessary to leverage the full feature-set of Ambassador Edge Stack as a traffic management and authentication tool. There are two options for configuring Ambassador, based on how you want to manage the configuration: Custom Resource Definitions (CRD's) and Annotations.

Ambassador primarily utilizes [Custom Resource Definitions](https://www.getambassador.io/docs/edge-stack/latest/topics/concepts/gitops-continuous-delivery/#policies-declarative-configuration-and-custom-resource-definitions) for managing configuration. These are custom `kind` resources that the Kubernetes API can read, and are used to update Ambassador's config in a way that is observable to the cluster as a whole (e.g. you can `kubectl get` these resources). These CRD's can be managed independent of the Seldon Deploy itself.

Ambassador also supports annotation based configuration, which can be applied to a Seldon Deployment using the `seldon.io/ambassador-config` annotation key. The overall formatting of the annotation-based config is the same as the CRD based config, except without the `metadata` and `spec` fields. The following snippets demonstrate the difference between CRD and Annotation based configurations, and are functionally identical.

```yaml
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
name: seldon_example_rest_mapping
spec:
prefix: /mycompany/ml/
service: production-model-example.seldon:8000
timeout_ms: 3000
```

* `seldon.io/ambassador-config:<configuration>` : The custom ambassador configuration
* Example: `"seldon.io/ambassador-config":"apiVersion: ambassador/v1\nkind: Mapping\nname: seldon_example_rest_mapping\nprefix: /mycompany/ml/\nservice: production-model-example.seldon:8000\ntimeout_ms: 3000"`
* Example: `"seldon.io/ambassador-config":"apiVersion: ambassador/v2\nkind: Mapping\nname: seldon_example_rest_mapping\nprefix: /mycompany/ml/\nservice: production-model-example.seldon:8000\ntimeout_ms: 3000"`

A worked example for [custom Ambassador config](../examples/ambassador_custom.html) is provided.
6 changes: 3 additions & 3 deletions examples/ambassador/custom/ambassador_custom.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@
"source": [
"## Setup Seldon Core\n",
"\n",
"Uset the setup notebook to [Setup Cluster](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Setup-Cluster) with [Ambassador Ingress](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Ambassador) and [Install Seldon Core](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Install-Seldon-Core). Instructions [also online](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html)."
"Use the setup notebook to [Setup Cluster](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Setup-Cluster) with [Ambassador Ingress](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Ambassador) and [Install Seldon Core](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html#Install-Seldon-Core). Instructions [also online](https://docs.seldon.io/projects/seldon-core/en/latest/examples/seldon_core_setup.html)."
]
},
{
Expand Down Expand Up @@ -97,7 +97,7 @@
" name: example-custom\n",
"spec:\n",
" annotations:\n",
" seldon.io/ambassador-config: 'apiVersion: ambassador/v1\n",
" seldon.io/ambassador-config: 'apiVersion: ambassador/v2\n",
"\n",
" kind: Mapping\n",
"\n",
Expand Down Expand Up @@ -294,4 +294,4 @@
},
"nbformat": 4,
"nbformat_minor": 4
}
}
73 changes: 70 additions & 3 deletions notebooks/seldon_core_setup.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@
"\n",
"[Ambassador install](https://docs.seldon.io/projects/seldon-core/en/latest/workflow/install.html#install-ambassador)\n",
"\n",
"**Note:** There are reported gRPC issues with ambassador (see https://github.com/SeldonIO/seldon-core/issues/473)."
"Ambassador Edge Stack:"
]
},
{
Expand Down Expand Up @@ -158,12 +158,49 @@
"outputs": [],
"source": [
"!helm install ambassador datawire/ambassador \\\n",
" --set image.repository=quay.io/datawire/ambassador \\\n",
" --set enableAES=false \\\n",
" --set image.repository=docker.io/datawire/ambassador \\\n",
" --set crds.keep=false \\\n",
" --namespace seldon-system"
]
},
{
"source": [
"Ambassador API Gateway (Emissary Ingress):"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!helm repo add datawire https://www.getambassador.io"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!helm repo update"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!helm install ambassador datawire/ambassador \\\n",
" --set image.repository=docker.io/datawire/ambassador \\\n",
" --set crds.keep=false \\\n",
" --set enableAES=false \\\n",
" --namespace seldon-system"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -180,6 +217,36 @@
"!kubectl rollout status deployment.apps/ambassador"
]
},
{
"source": [
"For Ambassador Edge Stack:"
],
"cell_type": "markdown",
"metadata": {}
},
{
"source": [
"**Note:** Absent configuration, Ambassador Edge Stack will attempt to use TLS with a self-signed cert. Either [set up TLS](https://www.getambassador.io/docs/edge-stack/latest/howtos/tls-termination/) or ignore certificate validation (i.e. `curl -k`)."
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!kubectl port-forward $(kubectl get pods -n seldon-system -l app.kubernetes.io/name=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon-system 8003:8443"
]
},
{
"source": [
"For Ambassador API Gateway:"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": null,
Expand Down

0 comments on commit 8249af5

Please sign in to comment.