Skip to content

Commit

Permalink
Merge pull request #2937 from RafalSkolasinski/issue/2821-init-contai…
Browse files Browse the repository at this point in the history 
…ner-customisation

Extend CRD: allow to define storageInitializerImage in the graph definition
  • Loading branch information
@seldondev
seldondev authored Feb 25, 2021
2 parents cc039b2 + aa49607 commit d045c39
Show file tree
Hide file tree
Showing 17 changed files with 293 additions and 118 deletions.
72 changes: 46 additions & 26 deletions doc/source/servers/overview.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,27 +49,41 @@ in our default [helm values](../charts/seldon-core-operator.html#values).
See the [Dockerfile](https://github.com/kubeflow/kfserving/blob/master/python/storage-initializer.Dockerfile
) and its [entrypoint](https://github.com/kubeflow/kfserving/blob/master/python/storage-initializer/scripts/initializer-entrypoint
) for a detailed reference.
You can overwrite this value to specify another default `initContainer`. See details on requirements bellow

Secrets are injected into the init containers as environmental variables from kubernetes `secrets`.
The default secret name can be defined by setting following [helm value](../charts/seldon-core-operator.html#values)
```yaml
predictiveUnit:
defaultEnvSecretRefName: ""
```

### Customizing Init Containers

One can specify a custom `Init Container` globally by overwriting the `storageInitializer.image` helm value as metnioned above.
The `entrypoint` of the `container` must take two arguments:
- first representing the models URI
- second the desired path where binary should be downloaded to
### Customizing Init Containers

To illustrate how `initContainers` are used by the prepackaged model servers, following effective `SeldonDeployment` with defualt `initContainer` details updated by the mutating webhook is provided:
You can specify a custom `initContainer` image and default `secret` **globally** by overwriting the helm values specified in the previous section.

To illustrate how `initContainers` are used by the prepackaged model servers, consider a Seldon Deployment with `volumes`, `volumeMounts` and `initContainers` injected by the `Seldon Core Operator`:
```yaml
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: example
spec:
name: iris
spec:
predictors:
- componentSpecs:
- name: default
replicas: 1
graph:
name: classifier
implementation: SKLEARN_SERVER
modelUri: s3://sklearn/iris
componentSpecs:
- spec:
volumes:
- name: classifier-provision-location
emptyDir: {}
containers:
- name: classifier
volumeMounts:
Expand All @@ -89,31 +103,37 @@ spec:
- secretRef:
name: seldon-init-container-secret
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /mnt/models
name: classifier-provision-location
```

volumes:
- emptyDir: {}
name: classifier-provision-location
Key observations:
- Our prepackaged model will expect model binaries to be saved into `/mnt/models` path
- Default `initContainers` name is constructed from `{predictiveUnitName}-model-initializer`
- The `entrypoint` of the `container` must take two arguments:
- First representing the models URI
- Second the desired path where binary should be downloaded to
- If user would to provide their own `initContainer` which name matches the above pattern it would be used as provided

Image and secret used by Storage Initializer can be customised per-deployment:
```yaml
apiVersion: machinelearning.seldon.io/v1
kind: SeldonDeployment
metadata:
name: rclone-sklearn
spec:
predictors:
- name: default
replicas: 1
graph:
children: []
implementation: SKLEARN_SERVER
modelUri: s3://sklearn/iris
name: classifier
name: defaul
implementation: SKLEARN_SERVER
modelUri: mys3:sklearn/iris
storageInitializerImage: gcr.io/kfserving/storage-initializer:v0.4.0 # Specify custom image here
envSecretRefName: seldon-init-container-secret # Specify custom secret here
```

Key observations:
- our prepackaged model will expect model binaries to be saved into `/mnt/models` path
- default `initContainers` name is constructed from `{predictiveUnitName}-model-initializer`

Currently image used for `initContainers` can only be specified globally via helm values.
The per deployment customisation is explored in this [GitHub issue #2611](https://github.com/SeldonIO/seldon-core/issues/2611).

See our [example](../examples/custom_init_container.html) to learn how to write a custom container that uses [rclone](https://rclone.org/) for cloud storage operations.

## Further Customisation for Prepackaged Model Servers
Expand Down
5 changes: 3 additions & 2 deletions examples/init_containers/Makefile
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
IMAGE_NAME=seldonio/rclone-init-container
IMAGE_VERSION := $(shell cat ../../version.txt)
IMAGE_NAME=seldonio/rclone-init-container-example
# IMAGE_VERSION := $(shell cat ../../version.txt)
IMAGE_VERSION=0.1

KIND_NAME ?= kind

Expand Down
120 changes: 91 additions & 29 deletions examples/init_containers/custom_init_container.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,9 +45,9 @@
"Added `gcs` successfully.\n",
"Bucket created successfully `minio-seldon/iris`.\n",
"`gcs/seldon-models/sklearn/iris/model.joblib` -> `minio-seldon/iris/model.joblib`\n",
"Total: 0 B, Transferred: 1.06 KiB, Speed: 8.57 KiB/s\n",
"Total: 0 B, Transferred: 1.06 KiB, Speed: 10.35 KiB/s\n",
"`gcs/seldon-models/sklearn/iris/metadata.yaml` -> `minio-seldon/iris/metadata.yaml`\n",
"Total: 0 B, Transferred: 162 B, Speed: 1.23 KiB/s\n"
"Total: 0 B, Transferred: 162 B, Speed: 1.35 KiB/s\n"
]
}
],
Expand All @@ -69,8 +69,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[2021-01-07 18:45:12 GMT] 162B metadata.yaml\n",
"[2021-01-07 18:45:11 GMT] 1.1KiB model.joblib\n"
"[2021-02-09 18:11:17 GMT] 162B metadata.yaml\n",
"[2021-02-09 18:11:16 GMT] 1.1KiB model.joblib\n"
]
}
],
Expand Down Expand Up @@ -308,7 +308,35 @@
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"data\": {\n",
" \"names\": [\n",
" \"t:0\",\n",
" \"t:1\",\n",
" \"t:2\"\n",
" ],\n",
" \"ndarray\": [\n",
" [\n",
" 0.9548873249364169,\n",
" 0.04505474761561406,\n",
" 5.7927447968952436e-05\n",
" ]\n",
" ]\n",
" },\n",
" \"meta\": {\n",
" \"requestPath\": {\n",
" \"classifier\": \"seldonio/sklearnserver:1.6.0-dev\"\n",
" }\n",
" }\n",
"}\n"
]
}
],
"source": [
"%%bash\n",
"curl -s -X POST -H 'Content-Type: application/json' \\\n",
Expand All @@ -320,9 +348,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Custom Init Container (as default container)\n",
"## Custom Init Container (the right way)\n",
"\n",
"It is also possible to prepare a custom init container image to:\n",
"- use it on an individual deployment\n",
"- set as a new default \n",
"\n",
"It is also possible to prepare a custom init container image and set it as default one. \n",
"For this purpose we need to build a docker image which entrypoint will accept two arguments:\n",
"- source\n",
"- destination\n",
Expand Down Expand Up @@ -357,7 +388,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This image is build and published as `seldonio/rclone-init-container`.\n",
"This image example is build and published as `seldonio/rclone-init-container-example`.\n",
"\n",
"`rclone` tool can be configured using both `rclone.conf` config file (as above) and environmental variables.\n",
"Note the remote name `mys3` in the name of environmental variables defined in the following secret:"
Expand Down Expand Up @@ -410,20 +441,6 @@
"!kubectl apply -f seldon-reclone-secret.yaml"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**IMPORTANT** For the following example to work Seldon Core need to be installed with following helm values\n",
"```yaml\n",
"storageInitializer:\n",
" image: seldonio/rclone-init-container:latest\n",
"\n",
"predictiveUnit:\n",
" defaultEnvSecretRefName: seldon-rclone-secret\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -454,13 +471,14 @@
"spec:\n",
" name: iris\n",
" predictors:\n",
" - graph:\n",
" children: []\n",
" - name: default\n",
" replicas: 1\n",
" graph:\n",
" name: classifier\n",
" implementation: SKLEARN_SERVER\n",
" modelUri: mys3:sklearn/iris\n",
" name: classifier\n",
" name: default\n",
" replicas: 1"
" storageInitializerImage: seldonio/rclone-init-container-example:0.1\n",
" envSecretRefName: seldon-rclone-secret "
]
},
{
Expand Down Expand Up @@ -502,14 +520,58 @@
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"data\": {\n",
" \"names\": [\n",
" \"t:0\",\n",
" \"t:1\",\n",
" \"t:2\"\n",
" ],\n",
" \"ndarray\": [\n",
" [\n",
" 0.9548873249364169,\n",
" 0.04505474761561406,\n",
" 5.7927447968952436e-05\n",
" ]\n",
" ]\n",
" },\n",
" \"meta\": {\n",
" \"requestPath\": {\n",
" \"classifier\": \"seldonio/sklearnserver:1.6.0-dev\"\n",
" }\n",
" }\n",
"}\n"
]
}
],
"source": [
"%%bash\n",
"curl -s -X POST -H 'Content-Type: application/json' \\\n",
" -d '{\"data\":{\"ndarray\":[[5.964, 4.006, 2.081, 1.031]]}}' \\\n",
" http://localhost:8003/seldon/seldon/rclone-as-default-init-container/api/v1.0/predictions | jq ."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Set new default storage initializer\n",
"\n",
"To user our newly created init container as default we need to configure `Seldon Core` installations by setting following helm values:\n",
"```yaml\n",
"storageInitializer:\n",
" image: seldonio/rclone-init-container-example:0.1\n",
"\n",
"predictiveUnit:\n",
" defaultEnvSecretRefName: seldon-rclone-secret\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand All @@ -519,7 +581,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 14,
"metadata": {},
"outputs": [
{
Expand Down
11 changes: 6 additions & 5 deletions examples/init_containers/rclone-default-init.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,11 @@ metadata:
spec:
name: iris
predictors:
- graph:
children: []
- name: default
replicas: 1
graph:
name: classifier
implementation: SKLEARN_SERVER
modelUri: mys3:sklearn/iris
name: classifier
name: default
replicas: 1
storageInitializerImage: seldonio/rclone-init-container-example:0.1
envSecretRefName: seldon-rclone-secret
4 changes: 4 additions & 0 deletions ...rator/templates/customresourcedefinition_seldondeployments.machinelearning.seldon.io.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4019,6 +4019,8 @@ spec:
type: string
serviceAccountName:
type: string
storageInitializerImage:
type: string
type:
type: string
type: object
Expand Down Expand Up @@ -4311,6 +4313,8 @@ spec:
type: array
serviceAccountName:
type: string
storageInitializerImage:
type: string
type:
type: string
required:
Expand Down
6 changes: 6 additions & 0 deletions ...or/templates/customresourcedefinition_v1_seldondeployments.machinelearning.seldon.io.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4882,6 +4882,8 @@ spec:
type: string
serviceAccountName:
type: string
storageInitializerImage:
type: string
type:
type: string
type: object
Expand Down Expand Up @@ -10650,6 +10652,8 @@ spec:
type: string
serviceAccountName:
type: string
storageInitializerImage:
type: string
type:
type: string
type: object
Expand Down Expand Up @@ -16418,6 +16422,8 @@ spec:
type: string
serviceAccountName:
type: string
storageInitializerImage:
type: string
type:
type: string
type: object
Expand Down
Loading

0 comments on commit d045c39

Please sign in to comment.