Skip to content

Issues: InftyAI/llmaz

Milestone v0.2.0
#259 opened Jan 27, 2025 by kerthcet
Open
[WebUI] Add support for webui
#160 opened Sep 13, 2024 by kerthcet
Open 3
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

35 Open 68 Closed
35 Open 68 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

PreStop command injection for inference engines feature Categorizes issue or PR as related to a new feature. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#288 opened Mar 4, 2025 by kerthcet
1 of 3 tasks
2
Lora Autoscaler feature Categorizes issue or PR as related to a new feature. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#287 opened Feb 28, 2025 by kerthcet
3 tasks done
v0.2.0
@kerthcet
2
Use client to verify the model inference is ready in E2E test feature Categorizes issue or PR as related to a new feature. help wanted Extra attention is needed needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#274 opened Feb 17, 2025 by kerthcet
3 tasks
@nayihz
3
Add popular open source models as in-tree support cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#268 opened Feb 14, 2025 by kerthcet
5 tasks
v0.2.0
1
Introduce inference reserve config for standby instance important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. needs-kind Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#265 opened Feb 8, 2025 by kerthcet
3 tasks done
v0.2.0
2
Milestone v0.2.0 needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#259 opened Jan 27, 2025 by kerthcet
2 tasks
vllm only has /health endpoint needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#241 opened Jan 17, 2025 by kerthcet
3 tasks
Support speculative decoding with llama.cpp feature Categorizes issue or PR as related to a new feature. help wanted Extra attention is needed needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#240 opened Jan 16, 2025 by kerthcet
3 tasks
4
Able to set the toRender parameters dynamically needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#239 opened Jan 16, 2025 by kerthcet
1 of 3 tasks
Unify the chat api for all inference servers feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#218 opened Dec 10, 2024 by kerthcet
3 tasks
2
⚠️ Action Required: Replace Deprecated gcr.io/kubebuilder/kube-rbac-proxy help wanted Extra attention is needed needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#211 opened Nov 30, 2024 by camilamacedo86
1
Add TensorRT-LLM support as another backend needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#205 opened Nov 18, 2024 by kerthcet
3 tasks
Install lws controller together with llmaz controller in the same namespace cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. help wanted Extra attention is needed needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#202 opened Nov 13, 2024 by kerthcet
3
Support speculative decoding with llama.cpp needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#197 opened Nov 5, 2024 by kerthcet
3 tasks
Serverless support needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#192 opened Oct 29, 2024 by kerthcet
3 tasks done
v0.2.0
1
Support to serving Stable Diffusion models needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#189 opened Oct 23, 2024 by kerthcet
1 of 3 tasks
Is there any early proposal or document about integrating with Gateway API ? feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#165 opened Sep 15, 2024 by caozhuozi
2
[WebUI] Add support for webui feature Categorizes issue or PR as related to a new feature. help wanted Extra attention is needed needs-kind Indicates a PR lacks a label and requires one. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#160 opened Sep 13, 2024 by kerthcet
3 tasks
v0.2.0
3
[Umbrella] Improve test coverages cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. good first issue Good for newcomers needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#156 opened Sep 12, 2024 by kerthcet
4
Support traditional models feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#133 opened Sep 9, 2024 by kerthcet
3 tasks done
1
Loading model weights more efficiently feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#119 opened Sep 2, 2024 by kerthcet
3 tasks
v0.2.0
@kerthcet
7
Support scaling with Spot instances for cost saving needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one. question Further information is requested
#106 opened Aug 27, 2024 by kerthcet
3 tasks
6
Support filesystems feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#100 opened Aug 19, 2024 by kerthcet
3 tasks done
1
Model aware scheduling feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#96 opened Aug 19, 2024 by kerthcet
2 of 3 tasks
3
Prompts managements feature Categorizes issue or PR as related to a new feature. needs-priority Indicates a PR lacks a label and requires one. needs-triage Indicates an issue or PR lacks a label and requires one.
#90 opened Aug 17, 2024 by kerthcet
3 tasks done
1
ProTip! What’s not been updated in a month: updated:<2025-02-05.