You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: doc/source/python/python_wrapping_docker.md
+42-34Lines changed: 42 additions & 34 deletions
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,19 @@
1
1
# Packaging a Python model for Seldon Core using Docker
2
2
3
-
4
3
In this guide, we illustrate the steps needed to wrap your own python model in a docker image ready for deployment with Seldon Core using Docker.
5
4
6
5
## Step 1 - Create your source code
7
6
8
7
You will need:
9
8
10
-
* A python file with a class that runs your model
11
-
* A requirements.txt with a seldon-core entry
9
+
- A python file with a class that runs your model
10
+
- A requirements.txt with a seldon-core entry
12
11
13
12
We will go into detail for each of these steps:
14
13
15
14
### Python file
16
-
Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at ```wrappers/s2i/python/test/model-template-app/MyModel.py```:
15
+
16
+
Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at `wrappers/s2i/python/test/model-template-app/MyModel.py`:
17
17
18
18
```python
19
19
classMyModel(object):
@@ -40,12 +40,13 @@ class MyModel(object):
40
40
return X
41
41
```
42
42
43
-
* The file is called MyModel.py and it defines a class MyModel
44
-
* The class contains a predict method that takes an array (numpy) X and feature_names and returns an array of predictions.
45
-
* You can add any required initialization inside the class init method.
46
-
* Your return array should be at least 2-dimensional.
43
+
- The file is called MyModel.py and it defines a class MyModel
44
+
- The class contains a predict method that takes an array (numpy) X and feature_names and returns an array of predictions.
45
+
- You can add any required initialization inside the class init method.
46
+
- Your return array should be at least 2-dimensional.
47
47
48
48
### requirements.txt
49
+
49
50
Populate a requirements.txt with any software dependencies your code requires. At a minimum the file should contain:
Use ```docker build . -t $ORG/$MODEL_NAME:$TAG``` to create your Docker image from source code. A simple name can be used but convention is to use the ORG/IMAGE:TAG format.
77
+
78
+
Use `docker build . -t $ORG/$MODEL_NAME:$TAG` to create your Docker image from source code. A simple name can be used but convention is to use the ORG/IMAGE:TAG format.
78
79
79
80
## Using with Keras/Tensorflow Models
80
81
81
82
To ensure Keras models with the Tensorflow backend work correctly you may need to call `_make_predict_function()` on your model after it is loaded. This is because Flask may call the prediction request in a separate thread from the one that initialised your model. See the [keras issue](https://github.com/keras-team/keras/issues/6462) for further discussion.
82
83
83
84
## Environment Variables
84
-
The required environment variables understood by the builder image are explained below. You can provide them in the Dockerfile or as `-e` parameters to `docker run`.
85
85
86
+
The required environment variables understood by the builder image are explained below. You can provide them in the Dockerfile or as `-e` parameters to `docker run`.
86
87
87
88
### MODEL_NAME
89
+
88
90
The name of the class containing the model. Also the name of the python file which will be imported.
89
91
90
92
### API_TYPE
@@ -95,55 +97,61 @@ API type to create. Can be REST or GRPC
95
97
96
98
The service type being created. Available options are:
97
99
98
-
* MODEL
99
-
* ROUTER
100
-
* TRANSFORMER
101
-
* COMBINER
102
-
* OUTLIER_DETECTOR
100
+
- MODEL
101
+
- ROUTER
102
+
- TRANSFORMER
103
+
- COMBINER
104
+
- OUTLIER_DETECTOR
103
105
104
106
### PERSISTENCE
105
107
106
108
Set either to 0 or 1. Default is 0. If set to 1 then your model will be saved periodically to redis and loaded from redis (if exists) or created fresh if not.
107
109
108
110
### FLASK_JSONIFY_PRETTYPRINT_REGULAR
109
111
110
-
Sets the flask application configuration `JSONIFY_PRETTYPRINT_REGULAR` for the REST API. Available options are `True`
112
+
Sets the flask application configuration `JSONIFY_PRETTYPRINT_REGULAR`(see [Configuration Handling - JSONIFY_PRETTYPRINT_REGULAR](https://flask.palletsprojects.com/config/#JSONIFY_PRETTYPRINT_REGULAR)) for the REST API. Available options are `True`
111
113
or `False`. If nothing is specified, flask's default value is used.
112
114
113
115
### FLASK_JSON_SORT_KEYS
114
116
115
-
Sets the flask application configuration `JSON_SORT_KEYS` for the REST API. Available options are `True` or `False`.
117
+
Sets the flask application configuration `JSON_SORT_KEYS` (see [Configuration Handling - JSON_SORT_KEYS](https://flask.palletsprojects.com/config/#JSON_SORT_KEYS)) for the REST API. Available options are `True` or `False`.
118
+
If nothing is specified, flask's default value is used.
119
+
120
+
### FLASK_PROPAGATE_EXCEPTIONS
121
+
122
+
Sets the flask application configuration `PROPAGATE_EXCEPTIONS` (see [Configuration Handling - PROPAGATE_EXCEPTIONS](https://flask.palletsprojects.com/config/#PROPAGATE_EXCEPTIONS)) for the REST API. Available options are `True` or `False`.
116
123
If nothing is specified, flask's default value is used.
117
124
118
125
## Creating different service types
119
126
120
127
### MODEL
121
128
122
-
*[A minimal skeleton for model source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/model-template-app)
123
-
*[Example model notebooks](../examples/notebooks.html)
129
+
-[A minimal skeleton for model source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/model-template-app)
130
+
-[Example model notebooks](../examples/notebooks.html)
124
131
125
132
### ROUTER
126
-
*[Description of routers in Seldon Core](../analytics/routers.html)
127
-
*[A minimal skeleton for router source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/router-template-app)
128
133
129
-
### TRANSFORMER
134
+
-[Description of routers in Seldon Core](../analytics/routers.html)
135
+
-[A minimal skeleton for router source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/router-template-app)
130
136
131
-
*[A minimal skeleton for transformer source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/transformer-template-app)
You can add arguments to your component which will be populated from the ```parameters``` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
145
+
146
+
You can add arguments to your component which will be populated from the `parameters` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
These arguments can be set when deploying in a Seldon Deployment. An example can be found in the [MNIST TFServing example](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/tfserving-mnist.ipynb) where the arguments are defined in the [SeldonDeployment](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/mnist_tfserving_deployment.json.template)which is partly show below:
154
+
These arguments can be set when deploying in a Seldon Deployment. An example can be found in the [MNIST TFServing example](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/tfserving-mnist.ipynb) where the arguments are defined in the [SeldonDeployment](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/mnist_tfserving_deployment.json.template) which is partly show below:
147
155
148
156
```
149
157
"graph": {
@@ -181,14 +189,13 @@ These arguments can be set when deploying in a Seldon Deployment. An example can
181
189
},
182
190
```
183
191
184
-
185
-
The allowable ```type``` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
186
-
192
+
The allowable `type` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
187
193
188
194
### Custom Metrics
189
-
```from version 0.3```
190
195
191
-
To add custom metrics to your response you can define an optional method ```metrics``` in your class that returns a list of metric dicts. An example is shown below:
196
+
`from version 0.3`
197
+
198
+
To add custom metrics to your response you can define an optional method `metrics` in your class that returns a list of metric dicts. An example is shown below:
192
199
193
200
```python
194
201
classMyModel(object):
@@ -205,9 +212,10 @@ For more details on custom metrics and the format of the metric dict see [here](
205
212
There is an [example notebook illustrating a model with custom metrics in python](../examples/custom_metrics.html).
206
213
207
214
### Custom Meta Data
208
-
```from version 0.3```
209
215
210
-
To add custom meta data you can add an optional method ```tags``` which can return a dict of custom meta tags as shown in the example below:
216
+
`from version 0.3`
217
+
218
+
To add custom meta data you can add an optional method `tags` which can return a dict of custom meta tags as shown in the example below:
0 commit comments