The Model Deployment Operator is a prototype system designed to automate the deployment and management of Nvidia Triton models on Kubernetes. By using configuration files stored in a Git repository, the system ensures that model deployments are consistent, version-controlled, and easily auditable. This approach simplifies the process of updating models, rolling back changes, and maintaining a history of deployments, making it easier to manage machine learning models at scale.
- Early Prototype: The project is in its early prototype stage and may not work as expected.
- Deployment Scripts: Scripts for deploying models are included, but more features and refinements are planned.
- Documentation: Basic documentation is available, but will be expanded as the project evolves.
- Custom CRD Generation: Uses Pydantic models to easily create and manage Custom Resource Definitions (CRDs) for Kubernetes.
- Triton Model Config JSON Schema: Provides a pre-built JSON schema for configuring Triton models, generated from the official model configuration protocol.
- Protobuf Conversion Tools: Includes tools to convert protobuf messages into JSON or YAML formats, making it easier to work with different data formats.
-
Clone the Repository:
git clone https://github.com/ogvalt/model-deployment-operator.git
-
Navigate to the Directory:
cd model-deployment-operator -
Follow the Examples: Check the
examplesdirectory for sample configurations and deployment scripts. -
Deploy Using Helm:
helm install model-deployment-operator ./helm
Contributions are welcome! Please open issues and pull requests to help improve this project.
- How to create CRD from pydantic model: