Skip to content

OpenVINO™ Model Server 2024.1
Compare
Choose a tag to compare
@dkalinowski dkalinowski released this 25 Apr 12:23
v2024.1
95b8b78
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

The 2024.1 has a few improvements in the serving functionality, demo enhancements and bug fixes.

Changes and improvements

  • Updated OpenVINO Runtime backend to 2024.1  Link

  • Added support for OpenVINO models with string data type on output. Together with the features introduced in 2024.0, now OVMS can support models with input and output of string type. That way you can take advantage of the tokenization built into the model as the first layer. You can also rely on any post-processing embedded into the model which returns just text. Check universal sentence encoder demo  and image classification with string output demo

  • Updated MediaPipe python calculators to support relative path for all related configuration and python code files. Now, the complete graph configuration folder can be deployed in arbitrary path without any code changes. It is demonstrated in the updated text generation demo.

  • Extended support for KServe REST API for MediaPipe graph endpoints. Now you can send the data in KServe JSON body. Check how it is used in text generation use case.

  • Added demo showcasing full RAG algorithm entirely delegated to the model server Link

  • Added RedHat UBI based Dockerfile for python demos, usage documented in python demos

Breaking changes

No breaking changes.

Bug fixes

  • Improvements in error handling for invalid requests and incorrect configuration
  • Fixes in the demos and documentation

You can use an OpenVINO Model Server public Docker images based on Ubuntu via the following command:
docker pull openvino/model_server:2024.1 - CPU device support with the image based on Ubuntu22.04
docker pull openvino/model_server:2024.1-gpu - GPU and CPU device support with the image based on Ubuntu22.04
or use provided binary packages.
The prebuilt image is available also on RedHat Ecosystem Catalog

Assets 6
Loading