diff --git a/docker/README.md b/docker/README.md index 35ffc18b43..a2379e0472 100644 --- a/docker/README.md +++ b/docker/README.md @@ -303,3 +303,7 @@ docker run --rm --shm-size=1g \ -p7071:7071 \ --mount type=bind,source=/path/to/model/store,target=/tmp/models torchserve --model-store=/tmp/models ``` + +# Example showing serving model using Docker container + +[This](../examples/image_classifier/mnist/Docker.md) is an example showing serving MNIST model using Docker. diff --git a/examples/README.md b/examples/README.md index 1bafd53e58..298c524d11 100644 --- a/examples/README.md +++ b/examples/README.md @@ -5,6 +5,7 @@ * [Creating mar file for an eager mode model](#creating-mar-file-for-eager-mode-model) * [Creating mar file for torchscript mode model](#creating-mar-file-for-torchscript-mode-model) * [Serving custom model with custom service handler](#serving-custom-model-with-custom-service-handler) +* [Serving model using Docker Container](image_classifier/mnist/Docker.md) * [Creating a Workflow](Workflows/dog_breed_classification) * [Custom Metrics](custom_metrics) * [Dynamic Batch Processing](image_classifier/resnet_152_batch) diff --git a/examples/image_classifier/mnist/Docker.md b/examples/image_classifier/mnist/Docker.md new file mode 100644 index 0000000000..ffd690e85e --- /dev/null +++ b/examples/image_classifier/mnist/Docker.md @@ -0,0 +1,54 @@ +# Digit recognition model with MNIST dataset using Docker container + +In this example, we show how to use a pre-trained custom MNIST model to performing real time Digit recognition with TorchServe. +We will be serving the model using a Docker container. + +The inference service would return the digit inferred by the model in the input image. + +We used the following pytorch example to train the basic MNIST model for digit recognition : +https://github.com/pytorch/examples/tree/master/mnist + +## Serve an MNIST model on TorchServe docker container + +Run the commands given in following steps from the parent directory of the root of the repository. For example, if you cloned the repository into /home/my_path/serve, run the steps from /home/my_path/serve + + ### Create a torch model archive using the torch-model-archiver utility to archive the above files. + + ```bash + torch-model-archiver --model-name mnist --version 1.0 --model-file examples/image_classifier/mnist/mnist.py --serialized-file examples/image_classifier/mnist/mnist_cnn.pt --handler examples/image_classifier/mnist/mnist_handler.py + ``` + + ### Move .mar file into model_store directory + + ```bash + mkdir model_store + mv mnist.mar model_store/ + ``` + + ### Start a docker container with torchserve + + ```bash + docker run --rm -it -p 8080:8080 -p 8081:8081 -p 8082:8082 -v $(pwd)/model_store:/home/model-server/model-store pytorch/torchserve:latest-cpu + ``` + + ### Register the model on TorchServe using the above model archive file + + ```bash + curl -X POST "localhost:8081/models?model_name=mnist&url=mnist.mar&initial_workers=4" + ``` + + If this succeeeds, you will see a message like below + + ```bash + { + "status": "Model \"mnist\" Version: 1.0 registered with 4 initial workers" + } + ``` + + ### Run digit recognition inference outside the container + + ```bash + curl http://127.0.0.1:8080/predictions/mnist -T examples/image_classifier/mnist/test_data/0.png + ``` + + The output in this case will be a `0`