Skip to content

OpenVINO™ Model Server 2023.3
Compare
Choose a tag to compare
@michalkulakowski michalkulakowski released this 24 Jan 13:55
· 3 commits to releases/2023/3 since this release
v2023.3
551f7ea
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

The 2023.3 is a major release with added a new feature and numerous improvements.

Changes and improvements

  • Included a set of new demos using custom nodes as a python code. They include LLM text generation, stable diffusion and seq2seq translation.

  • Improvements in the demo highlighting video stream analysis. A simple client example can now process the video stream from a local camera, video file or RTSP stream. The data can be sent to the model server via unary gRPC calls or gRPC streaming.

  • Changes in the public release artifacts – the base image of the public model server images is now updated to Ubuntu 22.04 and RHEL 8.8. Public docker images include support for python custom nodes but without custom python dependencies. The public binary distribution of the model server is targeted also on Ubuntu 22.04 and RHEL 8.8 but without python support (it can be deployed on bare metal hosts without python installed). Check building from source guide.

  • Improvements in the documentation https://docs.openvino.ai/2023.3/ovms_what_is_openvino_model_server.html

New Features (Preview)

  • Added support for serving MediaPipe graphs with custom nodes implemented as a python code. It greatly simplifies exposing GenAI algorithms based on Hugging Face and Optimum libraries. It can be also applied for arbitrary pre and post processing for the AI solutions. Learn more about it

Stable Feature

gRPC streaming support is out of preview and considered stable.

Breaking changes

No breaking changes.

Deprecation notices

Batch Size AUTO and Shape AUTO are deprecated and will be removed.
Use Dynamic Model Shape feature instead.

Bug fixes

  • OVMS handles boolean parameters to plugin config now #2197

  • Sporadic failures in the IrisTracking demo using gRPC stream are fixed #2161

  • Fixed handling of the incorrect MediaPipe graphs producing multiple outputs with the same name #2161

You can use an OpenVINO Model Server public Docker images based on Ubuntu via the following command:
docker pull openvino/model_server:2023.3 - CPU device support with the image based on Ubuntu22.04
docker pull openvino/model_server:2023.3-gpu - GPU and CPU device support with the image based on Ubuntu22.04
or use provided binary packages.
The prebuilt image is available also on RedHat Ecosystem Catalog

Assets 6
Loading