You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: doc/source/python/python_wrapping_docker.md
+46-50Lines changed: 46 additions & 50 deletions
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,19 @@
1
1
# Packaging a Python model for Seldon Core using Docker
2
2
3
+
3
4
In this guide, we illustrate the steps needed to wrap your own python model in a docker image ready for deployment with Seldon Core using Docker.
4
5
5
6
## Step 1 - Create your source code
6
7
7
8
You will need:
8
9
9
-
- A python file with a class that runs your model
10
-
- A requirements.txt with a seldon-core entry
10
+
* A python file with a class that runs your model
11
+
* A requirements.txt with a seldon-core entry
11
12
12
13
We will go into detail for each of these steps:
13
14
14
15
### Python file
15
-
16
-
Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at `wrappers/s2i/python/test/model-template-app/MyModel.py`:
16
+
Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at ```wrappers/s2i/python/test/model-template-app/MyModel.py```:
17
17
18
18
```python
19
19
classMyModel(object):
@@ -40,13 +40,12 @@ class MyModel(object):
40
40
return X
41
41
```
42
42
43
-
- The file is called MyModel.py and it defines a class MyModel
44
-
- The class contains a predict method that takes an array (numpy) X and feature_names and returns an array of predictions.
45
-
- You can add any required initialization inside the class init method.
46
-
- Your return array should be at least 2-dimensional.
43
+
* The file is called MyModel.py and it defines a class MyModel
44
+
* The class contains a predict method that takes an array (numpy) X and feature_names and returns an array of predictions.
45
+
* You can add any required initialization inside the class init method.
46
+
* Your return array should be at least 2-dimensional.
47
47
48
48
### requirements.txt
49
-
50
49
Populate a requirements.txt with any software dependencies your code requires. At a minimum the file should contain:
Use `docker build . -t $ORG/$MODEL_NAME:$TAG` to create your Docker image from source code. A simple name can be used but convention is to use the ORG/IMAGE:TAG format.
76
+
## Step 3 - Build your image
77
+
Use ```docker build . -t $ORG/$MODEL_NAME:$TAG``` to create your Docker image from source code. A simple name can be used but convention is to use the ORG/IMAGE:TAG format.
79
78
80
79
## Using with Keras/Tensorflow Models
81
80
82
81
To ensure Keras models with the Tensorflow backend work correctly you may need to call `_make_predict_function()` on your model after it is loaded. This is because Flask may call the prediction request in a separate thread from the one that initialised your model. See the [keras issue](https://github.com/keras-team/keras/issues/6462) for further discussion.
83
82
84
83
## Environment Variables
85
-
86
84
The required environment variables understood by the builder image are explained below. You can provide them in the Dockerfile or as `-e` parameters to `docker run`.
87
85
88
-
### MODEL_NAME
89
86
87
+
### MODEL_NAME
90
88
The name of the class containing the model. Also the name of the python file which will be imported.
91
89
92
90
### API_TYPE
@@ -97,11 +95,11 @@ API type to create. Can be REST or GRPC
97
95
98
96
The service type being created. Available options are:
99
97
100
-
- MODEL
101
-
- ROUTER
102
-
- TRANSFORMER
103
-
- COMBINER
104
-
- OUTLIER_DETECTOR
98
+
* MODEL
99
+
* ROUTER
100
+
* TRANSFORMER
101
+
* COMBINER
102
+
* OUTLIER_DETECTOR
105
103
106
104
### PERSISTENCE
107
105
@@ -111,51 +109,49 @@ Set either to 0 or 1. Default is 0. If set to 1 then your model will be saved pe
111
109
112
110
See [Flask - Builtin Configuration Values](https://flask.palletsprojects.com/config/#builtin-configuration-values) for possible configurations; the following are configurable when prefixed with the `FLASK_` string (e.g. `FLASK_JSON_SORT_KEYS` translates to `JSON_SORT_KEYS` in Flask):
113
111
114
-
- DEBUG
115
-
- EXPLAIN_TEMPLATE_LOADING
116
-
- JSONIFY_PRETTYPRINT_REGULAR
117
-
- JSON_SORT_KEYS
118
-
- PROPAGATE_EXCEPTIONS
119
-
- PRESERVE_CONTEXT_ON_EXCEPTION
120
-
- SESSION_COOKIE_HTTPONLY
121
-
- SESSION_COOKIE_SECURE
122
-
- SESSION_REFRESH_EACH_REQUEST
123
-
- TEMPLATES_AUTO_RELOAD
124
-
- TESTING
125
-
- TRAP_HTTP_EXCEPTIONS
126
-
- TRAP_BAD_REQUEST_ERRORS
127
-
- USE_X_SENDFILE
112
+
* DEBUG
113
+
* EXPLAIN_TEMPLATE_LOADING
114
+
* JSONIFY_PRETTYPRINT_REGULAR
115
+
* JSON_SORT_KEYS
116
+
* PROPAGATE_EXCEPTIONS
117
+
* PRESERVE_CONTEXT_ON_EXCEPTION
118
+
* SESSION_COOKIE_HTTPONLY
119
+
* SESSION_COOKIE_SECURE
120
+
* SESSION_REFRESH_EACH_REQUEST
121
+
* TEMPLATES_AUTO_RELOAD
122
+
* TESTING
123
+
* TRAP_HTTP_EXCEPTIONS
124
+
* TRAP_BAD_REQUEST_ERRORS
128
125
129
126
## Creating different service types
130
127
131
128
### MODEL
132
129
133
-
-[A minimal skeleton for model source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/model-template-app)
134
-
-[Example model notebooks](../examples/notebooks.html)
130
+
*[A minimal skeleton for model source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/model-template-app)
131
+
*[Example model notebooks](../examples/notebooks.html)
135
132
136
133
### ROUTER
137
-
138
-
-[Description of routers in Seldon Core](../analytics/routers.html)
139
-
-[A minimal skeleton for router source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/router-template-app)
134
+
*[Description of routers in Seldon Core](../analytics/routers.html)
135
+
*[A minimal skeleton for router source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/router-template-app)
140
136
141
137
### TRANSFORMER
142
138
143
-
-[A minimal skeleton for transformer source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/transformer-template-app)
You can add arguments to your component which will be populated from the `parameters` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
146
+
You can add arguments to your component which will be populated from the ```parameters``` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
These arguments can be set when deploying in a Seldon Deployment. An example can be found in the [MNIST TFServing example](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/tfserving-mnist.ipynb) where the arguments are defined in the [SeldonDeployment](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/mnist_tfserving_deployment.json.template) which is partly show below:
154
+
These arguments can be set when deploying in a Seldon Deployment. An example can be found in the [MNIST TFServing example](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/tfserving-mnist.ipynb) where the arguments are defined in the [SeldonDeployment](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/mnist_tfserving_deployment.json.template)which is partly show below:
159
155
160
156
```
161
157
"graph": {
@@ -193,13 +189,14 @@ These arguments can be set when deploying in a Seldon Deployment. An example can
193
189
},
194
190
```
195
191
196
-
The allowable `type` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
197
192
198
-
### Custom Metrics
193
+
The allowable ```type``` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
199
194
200
-
`from version 0.3`
201
195
202
-
To add custom metrics to your response you can define an optional method `metrics` in your class that returns a list of metric dicts. An example is shown below:
196
+
### Custom Metrics
197
+
```from version 0.3```
198
+
199
+
To add custom metrics to your response you can define an optional method ```metrics``` in your class that returns a list of metric dicts. An example is shown below:
203
200
204
201
```python
205
202
classMyModel(object):
@@ -215,11 +212,10 @@ For more details on custom metrics and the format of the metric dict see [here](
215
212
216
213
There is an [example notebook illustrating a model with custom metrics in python](../examples/custom_metrics.html).
217
214
218
-
### Custom Request Tags
219
-
220
-
`from version 0.3`
215
+
### Custom Meta Data
216
+
```from version 0.3```
221
217
222
-
To add custom request tags data you can add an optional method `tags` which can return a dict of custom meta tags as shown in the example below:
218
+
To add custom meta data you can add an optional method ```tags``` which can return a dict of custom meta tags as shown in the example below:
0 commit comments