Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug in compaction] Undefined variable in scheduler_adapter #43

Open
Alex-q-z opened this issue Dec 6, 2024 · 0 comments
Open

[Bug in compaction] Undefined variable in scheduler_adapter #43

Alex-q-z opened this issue Dec 6, 2024 · 0 comments

Comments

@Alex-q-z
Copy link
Contributor

Alex-q-z commented Dec 6, 2024

On the compact branch, in lmcache_vllm/scheduler_adapter.py, PreemptionMode is undefined on line 270. This will lead to a variable undefined error whenever a sequence group gets pre-empted, e.g. due to insufficient space for KV cache on GPU. To reproduce, try running any offline batch inference workload with a large batch size.
@Alex-q-z Alex-q-z changed the title [Bug in compaction] Undefined variable on line 270 [Bug in compaction] Undefined variable in scheduler_adapter Dec 6, 2024
Alex-q-z added a commit that referenced this issue Dec 9, 2024
@Alex-q-z
Add fix for issue #43
16f3e3e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Alex-q-z