Skip to content

Commit fea8b45

Browse files
author
Eric Meadows
committed
Propagate Flask Exceptions - useful to be configurable; updated documentation as well
1 parent 7edeb1d commit fea8b45

File tree

1 file changed

+42
-34
lines changed

1 file changed

+42
-34
lines changed

doc/source/python/python_wrapping_docker.md

Lines changed: 42 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,19 @@
11
# Packaging a Python model for Seldon Core using Docker
22

3-
43
In this guide, we illustrate the steps needed to wrap your own python model in a docker image ready for deployment with Seldon Core using Docker.
54

65
## Step 1 - Create your source code
76

87
You will need:
98

10-
* A python file with a class that runs your model
11-
* A requirements.txt with a seldon-core entry
9+
- A python file with a class that runs your model
10+
- A requirements.txt with a seldon-core entry
1211

1312
We will go into detail for each of these steps:
1413

1514
### Python file
16-
Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at ```wrappers/s2i/python/test/model-template-app/MyModel.py```:
15+
16+
Your source code should contain a python file which defines a class of the same name as the file. For example, looking at our skeleton python model file at `wrappers/s2i/python/test/model-template-app/MyModel.py`:
1717

1818
```python
1919
class MyModel(object):
@@ -40,12 +40,13 @@ class MyModel(object):
4040
return X
4141
```
4242

43-
* The file is called MyModel.py and it defines a class MyModel
44-
* The class contains a predict method that takes an array (numpy) X and feature_names and returns an array of predictions.
45-
* You can add any required initialization inside the class init method.
46-
* Your return array should be at least 2-dimensional.
43+
- The file is called MyModel.py and it defines a class MyModel
44+
- The class contains a predict method that takes an array (numpy) X and feature_names and returns an array of predictions.
45+
- You can add any required initialization inside the class init method.
46+
- Your return array should be at least 2-dimensional.
4747

4848
### requirements.txt
49+
4950
Populate a requirements.txt with any software dependencies your code requires. At a minimum the file should contain:
5051

5152
```
@@ -72,19 +73,20 @@ ENV PERSISTENCE 0
7273
CMD exec seldon-core-microservice $MODEL_NAME $API_TYPE --service-type $SERVICE_TYPE --persistence $PERSISTENCE
7374
```
7475

75-
7676
## Step 3 - Build your image
77-
Use ```docker build . -t $ORG/$MODEL_NAME:$TAG``` to create your Docker image from source code. A simple name can be used but convention is to use the ORG/IMAGE:TAG format.
77+
78+
Use `docker build . -t $ORG/$MODEL_NAME:$TAG` to create your Docker image from source code. A simple name can be used but convention is to use the ORG/IMAGE:TAG format.
7879

7980
## Using with Keras/Tensorflow Models
8081

8182
To ensure Keras models with the Tensorflow backend work correctly you may need to call `_make_predict_function()` on your model after it is loaded. This is because Flask may call the prediction request in a separate thread from the one that initialised your model. See the [keras issue](https://github.com/keras-team/keras/issues/6462) for further discussion.
8283

8384
## Environment Variables
84-
The required environment variables understood by the builder image are explained below. You can provide them in the Dockerfile or as `-e` parameters to `docker run`.
8585

86+
The required environment variables understood by the builder image are explained below. You can provide them in the Dockerfile or as `-e` parameters to `docker run`.
8687

8788
### MODEL_NAME
89+
8890
The name of the class containing the model. Also the name of the python file which will be imported.
8991

9092
### API_TYPE
@@ -95,55 +97,61 @@ API type to create. Can be REST or GRPC
9597

9698
The service type being created. Available options are:
9799

98-
* MODEL
99-
* ROUTER
100-
* TRANSFORMER
101-
* COMBINER
102-
* OUTLIER_DETECTOR
100+
- MODEL
101+
- ROUTER
102+
- TRANSFORMER
103+
- COMBINER
104+
- OUTLIER_DETECTOR
103105

104106
### PERSISTENCE
105107

106108
Set either to 0 or 1. Default is 0. If set to 1 then your model will be saved periodically to redis and loaded from redis (if exists) or created fresh if not.
107109

108110
### FLASK_JSONIFY_PRETTYPRINT_REGULAR
109111

110-
Sets the flask application configuration `JSONIFY_PRETTYPRINT_REGULAR` for the REST API. Available options are `True`
112+
Sets the flask application configuration `JSONIFY_PRETTYPRINT_REGULAR` (see [Configuration Handling - JSONIFY_PRETTYPRINT_REGULAR](https://flask.palletsprojects.com/config/#JSONIFY_PRETTYPRINT_REGULAR)) for the REST API. Available options are `True`
111113
or `False`. If nothing is specified, flask's default value is used.
112114

113115
### FLASK_JSON_SORT_KEYS
114116

115-
Sets the flask application configuration `JSON_SORT_KEYS` for the REST API. Available options are `True` or `False`.
117+
Sets the flask application configuration `JSON_SORT_KEYS` (see [Configuration Handling - JSON_SORT_KEYS](https://flask.palletsprojects.com/config/#JSON_SORT_KEYS)) for the REST API. Available options are `True` or `False`.
118+
If nothing is specified, flask's default value is used.
119+
120+
### FLASK_PROPAGATE_EXCEPTIONS
121+
122+
Sets the flask application configuration `PROPAGATE_EXCEPTIONS` (see [Configuration Handling - PROPAGATE_EXCEPTIONS](https://flask.palletsprojects.com/config/#PROPAGATE_EXCEPTIONS)) for the REST API. Available options are `True` or `False`.
116123
If nothing is specified, flask's default value is used.
117124

118125
## Creating different service types
119126

120127
### MODEL
121128

122-
* [A minimal skeleton for model source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/model-template-app)
123-
* [Example model notebooks](../examples/notebooks.html)
129+
- [A minimal skeleton for model source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/model-template-app)
130+
- [Example model notebooks](../examples/notebooks.html)
124131

125132
### ROUTER
126-
* [Description of routers in Seldon Core](../analytics/routers.html)
127-
* [A minimal skeleton for router source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/router-template-app)
128133

129-
### TRANSFORMER
134+
- [Description of routers in Seldon Core](../analytics/routers.html)
135+
- [A minimal skeleton for router source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/router-template-app)
130136

131-
* [A minimal skeleton for transformer source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/transformer-template-app)
132-
* [Example transformers](https://github.com/SeldonIO/seldon-core/tree/master/examples/transformers)
137+
### TRANSFORMER
133138

139+
- [A minimal skeleton for transformer source code](https://github.com/SeldonIO/seldon-core/tree/master/wrappers/s2i/python/test/transformer-template-app)
140+
- [Example transformers](https://github.com/SeldonIO/seldon-core/tree/master/examples/transformers)
134141

135142
## Advanced Usage
136143

137144
### Model Class Arguments
138-
You can add arguments to your component which will be populated from the ```parameters``` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
145+
146+
You can add arguments to your component which will be populated from the `parameters` defined in the SeldonDeloyment when you deploy your image on Kubernetes. For example, our [Python TFServing proxy](https://github.com/SeldonIO/seldon-core/tree/master/integrations/tfserving) has the class init method signature defined as below:
139147

140148
```python
141149
class TfServingProxy(object):
142150

143151
def __init__(self,rest_endpoint=None,grpc_endpoint=None,model_name=None,signature_name=None,model_input=None,model_output=None):
144152
```
145153

146-
These arguments can be set when deploying in a Seldon Deployment. An example can be found in the [MNIST TFServing example](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/tfserving-mnist.ipynb) where the arguments are defined in the [SeldonDeployment](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/mnist_tfserving_deployment.json.template) which is partly show below:
154+
These arguments can be set when deploying in a Seldon Deployment. An example can be found in the [MNIST TFServing example](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/tfserving-mnist.ipynb) where the arguments are defined in the [SeldonDeployment](https://github.com/SeldonIO/seldon-core/blob/master/examples/models/tfserving-mnist/mnist_tfserving_deployment.json.template) which is partly show below:
147155

148156
```
149157
"graph": {
@@ -181,14 +189,13 @@ These arguments can be set when deploying in a Seldon Deployment. An example can
181189
},
182190
```
183191

184-
185-
The allowable ```type``` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
186-
192+
The allowable `type` values for the parameters are defined in the [proto buffer definition](https://github.com/SeldonIO/seldon-core/blob/44f7048efd0f6be80a857875058d23efc4221205/proto/seldon_deployment.proto#L117-L131).
187193

188194
### Custom Metrics
189-
```from version 0.3```
190195

191-
To add custom metrics to your response you can define an optional method ```metrics``` in your class that returns a list of metric dicts. An example is shown below:
196+
`from version 0.3`
197+
198+
To add custom metrics to your response you can define an optional method `metrics` in your class that returns a list of metric dicts. An example is shown below:
192199

193200
```python
194201
class MyModel(object):
@@ -205,9 +212,10 @@ For more details on custom metrics and the format of the metric dict see [here](
205212
There is an [example notebook illustrating a model with custom metrics in python](../examples/custom_metrics.html).
206213

207214
### Custom Meta Data
208-
```from version 0.3```
209215

210-
To add custom meta data you can add an optional method ```tags``` which can return a dict of custom meta tags as shown in the example below:
216+
`from version 0.3`
217+
218+
To add custom meta data you can add an optional method `tags` which can return a dict of custom meta tags as shown in the example below:
211219

212220
```python
213221
class MyModel(object):

0 commit comments

Comments
 (0)