Example of training and serving a simple sequential model in TF using TFX in docker
Also create a simple FLASK API endpoint to query model's metadata and for predictions
Build and train model first using train.py. This builds and trains a very basic convnet using fashion mnist dataset
Run make start-model to start TFX server
Run make start-api to start api endpoint
Query the api for model's status:
curl -XGET http://localhost:5000/model/fashion_models
Query the api for model's metadata:
curl -XGET http://localhost:5000/model/fashion_models/metadata
To test a prediction, use sample_images.py to generate some test images for predictions using the fashion mnist test set:
python sample_images.py --num 10
It will create a local dir of testimgs
To make a prediction, you can use curl to POST the image to the API endpoint:
For REST endpoint:
curl -XPOST -F 'data=@./testimgs/test_0_label_9.png' http://localhost:5000/models/fashion_models/predict
For GRPC endpoint:
curl -XPOST -F 'data=@./testimgs/test_0_label_9.png' http://localhost:5000/models/fashion_models/predict_grpc
An example response would be:
The model thought this was a Ankle boot (class 9), and it was actually a Ankle boot (class 9)
Different port numbers expose different prediction types:
GRPC => 8500
Rest => 8501
To run locally using docker:
docker run -it --rm -p 8501:8501 -v "$ML_PATH:/models/fashion_models" -e MODEL_NAME=fashion_models tensorflow/serving