[V1][Scheduler] Avoid calling _try_schedule_encoder_inputs for every request
#15778
Conversation
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
WoosukKwon
requested review from
alexm-redhat,
comaniac,
njhill,
robertgshaw2-redhat and
ywang96
as code owners
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
WoosukKwon
changed the title
_try_schedule_encoder_inputs for every request
…y request (vllm-project#15778) Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu> Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
…y request (vllm-project#15778) Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu> Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>
…y request (vllm-project#15778) Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
…y request (vllm-project#15778) Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
…y request (vllm-project#15778) Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu> Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
Currently, we call
_try_schedule_encoder_inputsfor every request and every step.This PR avoids this function call by only calling it when the request includes encoder inputs (e.g., images).