add keda example

examples/keda/keda_prom_auto_scale.ipynb

@@ -0,0 +1,344 @@
+{
+ "metadata": {
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.7-final"
+  },
+  "orig_nbformat": 2,
+  "kernelspec": {
+   "name": "python3",
+   "display_name": "Python 3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2,
+ "cells": [
+  {
+   "source": [
+    "\n",
+    "# Scale Seldon Deployments based on Prometheus Metrics.\n",
+    "This notebook shows how you can scale Seldon Deployments based on Prometheus metrics via KEDA. \n",
+    "\n",
+    "[KEDA](https://keda.sh/) is a Kubernetes-based Event Driven Autoscaler. With KEDA, you can drive the scaling of any container in Kubernetes based on the number of events needing to be processed. \n",
+    "\n",
+    "With the support of KEDA in Seldon, you can scale your seldon deployments with any scalers listed [here](https://keda.sh/docs/2.0/scalers/).\n",
+    "In this example we will scale the seldon deployment with Prometheus metrics as an example."
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "source": [
+    "## Install Seldon Core\n",
+    "\n",
+    "Follow the [Seldon Core Install documentation](https://docs.seldon.io/projects/seldon-core/en/latest/workflow/install.html)."
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl create namespace seldon-system"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!helm install seldon-core seldon-core-operator --repo https://storage.googleapis.com/seldon-charts --namespace seldon-system"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl get pod -n seldon-system"
+   ]
+  },
+  {
+   "source": [
+    "## Install Seldon Core Analytics"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "source": [
+    "seldon-core-analytics contains Prometheus and Grafana installation with a basic Grafana dashboard showing the default Prometheus metrics exposed by Seldon for each inference graph deployed. \n",
+    "Later we will use the Prometheus service installed to provide metrics in order to scale the Seldon models."
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!helm install seldon-core-analytics seldon-core-analytics --repo https://storage.googleapis.com/seldon-charts --namespace seldon-system"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl get pod -n seldon-system"
+   ]
+  },
+  {
+   "source": [
+    "## Install KEDA"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "source": [
+    "!kubectl apply -f https://github.com/kedacore/keda/releases/download/v2.0.0-beta/keda-2.0.0-beta.yaml"
+   ],
+   "cell_type": "code",
+   "metadata": {
+    "tags": []
+   },
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl get pod -n keda"
+   ]
+  },
+  {
+   "source": [
+    "## Create model with KEDA"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "source": [
+    "To create a model with KEDA autoscaling you just need to add a KEDA spec refering in the Deployment, e.g.:\n",
+    "```yaml\n",
+    "kedaSpec:\n",
+    "  pollingInterval: 15                                # Optional. Default: 30 seconds\n",
+    "  minReplicaCount: 1                                 # Optional. Default: 0\n",
+    "  maxReplicaCount: 5                                 # Optional. Default: 100\n",
+    "  triggers:\n",
+    "  - type: prometheus\n",
+    "    metadata:\n",
+    "      # Required\n",
+    "      serverAddress: http://seldon-core-analytics-prometheus-seldon.seldon-system.svc.cluster.local\n",
+    "      metricName: access_frequency\n",
+    "      threshold: '10'\n",
+    "      query: rate(seldon_api_executor_client_requests_seconds_count{seldon_app=~\"seldon-model-example\"}[10s]\n",
+    "```\n",
+    "The full SeldonDeployment spec is shown below."
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!pygmentize model_with_keda_prom.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl create -f model_with_keda_prom.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl rollout status deploy/$(kubectl get deploy -l seldon-deployment-id=seldon-model -o jsonpath='{.items[0].metadata.name}')"
+   ]
+  },
+  {
+   "source": [
+    "## Create Load"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "source": [
+    "We label some nodes for the loadtester. We attempt the first two as for Kind the first node shown will be the master."
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "source": [
+    "!kubectl label nodes $(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') role=locust\n",
+    "!kubectl label nodes $(kubectl get nodes -o jsonpath='{.items[1].metadata.name}') role=locust"
+   ],
+   "cell_type": "code",
+   "metadata": {
+    "tags": []
+   },
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "source": [
+    "Before add loads to the model, there is only one replica"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl get deployment seldon-model-example-0-classifier"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!helm install seldon-core-loadtesting seldon-core-loadtesting --repo https://storage.googleapis.com/seldon-charts \\\n",
+    "    --set locust.host=http://seldon-model-example:8000 \\\n",
+    "    --set oauth.enabled=false \\\n",
+    "    --set locust.hatchRate=1 \\\n",
+    "    --set locust.clients=1 \\\n",
+    "    --set loadtest.sendFeedback=0 \\\n",
+    "    --set locust.minWait=0 \\\n",
+    "    --set locust.maxWait=0 \\\n",
+    "    --set replicaCount=1"
+   ]
+  },
+  {
+   "source": [
+    "After a few mins you should see the deployment scaled to 5 replicas"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl get deployment/seldon-model-example-0-classifier scaledobject/seldon-model-example-0-classifier"
+   ]
+  },
+  {
+   "source": [
+    "## Remove Load"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!helm delete seldon-core-loadtesting"
+   ]
+  },
+  {
+   "source": [
+    "After 5-10 mins you should see the deployment replica number decrease to 1"
+   ],
+   "cell_type": "markdown",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl get pods,deployments,hpa,scaledobject"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "!kubectl delete -f model_with_keda_prom.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ]
+}

examples/keda/model_with_keda_prom.yaml

@@ -0,0 +1,37 @@
+
+apiVersion: machinelearning.seldon.io/v1
+kind: SeldonDeployment
+metadata:
+  name: seldon-model
+spec:
+  name: test-deployment
+  predictors:
+  - componentSpecs:
+    - spec:
+        containers:
+        - image: seldonio/mock_classifier_rest:1.3
+          imagePullPolicy: IfNotPresent
+          name: classifier
+          resources:
+            requests:
+              cpu: '0.5'
+      kedaSpec:
+        pollingInterval: 15                                # Optional. Default: 30 seconds
+        minReplicaCount: 1                                 # Optional. Default: 0
+        maxReplicaCount: 5                                 # Optional. Default: 100
+        triggers:
+        - type: prometheus
+          metadata:
+            # Required
+            serverAddress: http://seldon-core-analytics-prometheus-seldon.seldon-system.svc.cluster.local
+            metricName: access_frequency
+            threshold: '10'
+            query: rate(seldon_api_executor_client_requests_seconds_count{seldon_app=~"seldon-model-example"}[10s])
+        terminationGracePeriodSeconds: 1
+    graph:
+      children: []
+      endpoint:
+        type: REST
+      name: classifier
+      type: MODEL
+    name: example

operator/Dockerfile.redhat

@@ -1,5 +1,5 @@
 # Build the manager binary
-FROM golang:1.13.2 as builder
+FROM golang:1.14.9 as builder
 
 WORKDIR /workspace
 # Copy the Go Modules manifests