Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Umbrella] Add webhook for validation #710

Open
7 tasks
kerthcet opened this issue Feb 19, 2025 · 2 comments
Open
7 tasks

[Umbrella] Add webhook for validation #710

kerthcet opened this issue Feb 19, 2025 · 2 comments
Labels
kind/enhancement New feature or request priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone

Comments

@kerthcet
Copy link
Collaborator

kerthcet commented Feb 19, 2025

🚀 Feature Description and Motivation

Webhook is used for CRD validations, and will fail fast compared to runtime validation.

Use Case

Once CRD is not right configured, fail fast.

  • webhook framework
  • add integration tests to CI && separate with E2E tests
  • ModelAdapter
  • PodAutoscaler
  • KVCache
  • RayclusterFleet
  • RayclusterReplicaset

Proposed Solution

No response

@kerthcet
Copy link
Collaborator Author

/assign

@kerthcet kerthcet changed the title Add webhook for validation [Umbrella] Add webhook for validation Feb 19, 2025
@Jeffwan
Copy link
Collaborator

Jeffwan commented Feb 25, 2025

We can discuss more details on the webhook usage. In the examples, we just use huggingface models for simplicity. However, in real world, most users has to fetch weights from S3 like object storage.

The challenge at this moment is AIBRix doesn't have any orchestration support to hide those details like llamaz or kubeAI.
As a mid term solution. I am thinking whether we can leverage webhook to convert more model configuration from annotations to specs fields like inject sidecar container for model downloading etc. that will fill the gap of missing model orchestration.

@Jeffwan Jeffwan added kind/enhancement New feature or request priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Feb 25, 2025
@Jeffwan Jeffwan added this to the v0.3.0 milestone Feb 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or request priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Development

No branches or pull requests

2 participants