Skip to content

OpenVINO™ Model Server 2023.1
Compare
Choose a tag to compare
@atobiszei atobiszei released this 18 Sep 13:17
· 5 commits to releases/2023/1 since this release
v2023.1
f36c885
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

OpenVINO™ Model Server 2023.1

The 2023.1 is a major release with numerous improvements and changes.

New Features

  • Improvements in Model Server with MediaPipe integration. In the previous version MediaPipe scheduler was included in OpenVINO Model Server as a preview. Now, the MediaPipe graph scheduler is added by default and officially supported. Check mediapipe in the model server documentation. This release includes the following improvements in running requests calls to the graphs:
    • GetModelMetadata implementation for MediaPipe graphs – the calls to model metadata returns information about the expected inputs and outputs names from the graph with the limitation on shape and datatype
    • Support for data serialization and deserialization to a range of types: ov::Tensor, mediapipe::Image, KServe ModelInfer Request/Response – those capabilities simplify adoption of the existing graphs which might have on the input and output the expected data in many different formats. Now the data submitted to the KServe endpoint can be automatically deserialized to the expected type. The deserialization function is determined based on the naming convention in the graph input and output tags in the graphs config. Check more details.
    • OpenVINOInferenceCalculator support for a range of input formats from ov::Tensor to tensorflow::Tensor and TfLite::Tensor - the OpenVINOInferenceCalculator has been created as a replacement for Tensorflow calculators. It can accept the input data and returns the data with a range of possible formats. That simplifies just swapping inference related nodes in the existing graphs without changing the rest of the graph. Learn more about the calculators
    • Added demos based on MediaPipe upstream graphs: holistic sensory analysis, object detection, iris detection
  • Improvements in C-API interface:
    • Added OVMS_ApiVersion call
    • Added support for C-API calls to DAG pipelines
    • Changed data type in API calls for data shape from uint64_t to int64_t and dimCount from uint32_t to size_t, this is breaking change
    • Added a call to servable (model, DAG) metadata and state
    • Added a call to get ServerMetadata
  • Improvements in error handling
  • Improvements in GRPC and REST status codes - the error statuses will include more meaningful and accurate info about the culprit
  • Support for models with scalars on input (empty shape) - model server can be used with models even with input shape represented by an empty list [] (scalar).
  • Support for input with zero size dimensions - model server can now accept requests to dynamic shape models even with 0 size like [0,234]
  • Added support for TFLite models - OpenVINO Model Server can not directly serve models with .tflite extension
  • Demo improvements:

Breaking changes

  • Changed few of the C-API functions names. Check this commit

Bug fixes

  • Fix REST status code when the improper path is requested
  • metrics endpoint now returns correct response even with unsupported parameters

You can use an OpenVINO Model Server public Docker image's based on Ubuntu via the following command:
docker pull openvino/model_server:2023.1  - CPU device support with the image based on Ubuntu20.04
docker pull openvino/model_server:2023.1-gpu - GPU and CPU device support with the image based on Ubuntu22.04
or use provided binary packages.
The prebuilt image is available also on RedHat Ecosystem Catalog

Assets 8
Loading