OpenVINO™ Model Server 2021.4.2

The 2021.4.2 version is a hotfix release for the OpenVINO Model Server. It includes a few bug fixes and enhancements in the exemplary clients.

Bug fixes:

• Fixed an issue with inference execution on NCS stick which allows loading multiple models at the same time. Now, with the config mode, multiple models can be passed to NCS device via a parameter --target_device MYRIAD.
• Documented docker container deployment with NCS stick without the privileged mode.
• Fixed handing of parameters including nested double quote" characters in the startup options in the docker container with an nginx proxy. It was impacting parameters like --plugin_config '{"CPU_THROUGHPUT_STREAMS":"1"}'
• Improved handling of OpenVINO plugin config parameters. Previously wrong type for a plugin parameter value didn’t return an error so it was easy to miss the fact that the parameter was ignored. Now the device plugin configuration will accept numerical values both with and without quotes. --plugin_config '{"CPU_THROUGHPUT_STREAMS":"1"}' and --plugin_config '{"CPU_THROUGHPUT_STREAMS":1}' are fine now. Invalid format of the value will raise an error.
• The parameters for changing the layout and shape with multiple inputs/outputs, will use the updated model tensor name as defined in the mapping_config.json file. It refers to a format like {"input1":"NHWC","input2":"NHWC"}
• External contribution to a custom node model_zoo_intel_object_detection - added labels output in the Directed Acyclic Graph Scheduler custom node. Now, it includes in the output also the labels from an object detection model.
• Security related updates

Exemplary client’s improvements:

• OVMS Demo with Bert model – question answering python application
• C++ async client example – it demonstrates how to connect to the model server using C++ application, but it can also be a convenient tool to test OVMS performance with the execution concurrency.
• Golang client – a demonstration how to connect to OVMS via gRPC protocol from a Golang application
• Updated Optical Character Recognition pipeline example to use a combination of EAST-REST50 model with a text recognition model from OpenVINO Model Zoo

You can use an OpenVINO Model Server public Docker image based on Ubuntu via the following command:
docker pull openvino/model_server:2021.4.2 or
docker pull openvino/model_server:2021.4.2-gpu