Skip to content

Commit

Permalink
Fixed log message and updated the doc
Browse files Browse the repository at this point in the history
  • Loading branch information
taavi-primer authored and seldondev committed Apr 30, 2020
1 parent 76fd1d4 commit 76902e1
Show file tree
Hide file tree
Showing 2 changed files with 35 additions and 1 deletion.
34 changes: 34 additions & 0 deletions doc/source/python/python_component.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,40 @@ class DeepMnist(object):
return predictions.astype(np.float64)
```
### Single-threaded Flask for REST (experimental)
To run your class single-threaded with Flask set the environment variable `FLASK_SINGLE_THREADED` to 1. This will set the `threaded` parameter of the Flask app to `False`. It is not the optimal setup for most models, but can be useful when your model cannot be made thread-safe like many GPU-based models that deadlock when accessed from multiple threads.
```
apiVersion: machinelearning.seldon.io/v1alpha2
kind: SeldonDeployment
metadata:
name: flaskexample
spec:
name: worker
predictors:
- componentSpecs:
- spec:
containers:
- image: seldonio/mock_classifier:1.0
name: classifier
env:
- name: FLASK_SINGLE_THREADED
value: '1'
terminationGracePeriodSeconds: 1
graph:
children: []
endpoint:
type: REST
name: classifier
type: MODEL
labels:
version: v1
name: example
replicas: 1

```
## Multi-value numpy arrays
By default, when using the data ndarray parameter, the conversion to ndarray (by default) converts all inner types into the same type. With models that may take as input arrays with different value types, you will be able to do so by overriding the `predict_raw` function yourself which gives you access to the raw request, and creating the numpy array as follows:
Expand Down
2 changes: 1 addition & 1 deletion python/seldon_core/microservice.py
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ def rest_prediction_server():
)

logger.info(
"REST gunicorn microservice running on port %i single-threaded=%s",
"REST microservice running on port %i single-threaded=%s",
port,
args.single_threaded,
)
Expand Down

0 comments on commit 76902e1

Please sign in to comment.