[V1] Get supported tasks from model runner instead of model config #21585
Conversation
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
DarkLight1337
added
the
ready
DarkLight1337
requested review from
WoosukKwon,
aarnphm,
alexm-redhat,
comaniac,
njhill,
robertgshaw2-redhat,
youkaichao,
ywang96 and
zhuohan123
as code owners
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors how supported tasks are determined by moving the logic from the model configuration to the model runner. This is a positive change as it makes the model implementation the single source of truth for its capabilities. The changes are applied consistently across the codebase, covering both V0 and V1 execution paths and various entrypoints like the API server and batch runner. The introduction of a new vllm/tasks.py file to centralize task definitions is a good step towards better code organization. The implementation appears correct and robust, and I did not identify any high or critical severity issues.
Mirroring changes from vllm-project/vllm#21585 to HPU code Signed-off-by: Konrad Zawora <kzawora@habana.ai>
# Changes - Remove prompt adapter config - Based on upstream vllm changes in vllm-project/vllm#20588 - Implement `get_supported_tasks` in model_runner for online API - only needed for online API, related PR - vllm-project/vllm#21585 Implementation for online API: ``` if envs.VLLM_USE_V1: supported_tasks = await engine_client \ .get_supported_tasks() # type: ignore else: supported_tasks = model_config.supported_tasks ``` ## Related Issues fix #336 --------- Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: x22x22 <wadeking@qq.com>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Paul Pak <paulpak58@gmail.com>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Diego-Castan <diego.castan@ibm.com>
Mirroring changes from vllm-project/vllm#21585 to HPU code Signed-off-by: Konrad Zawora <kzawora@habana.ai> Signed-off-by: Thomas Atta-fosu <tattafosu@habana.ai>
…llm-project#21585) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
3 participants
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.Purpose
Follow-up to #21227
Since the supported tasks are defined on the model instance, We should get them from the model runner, which has access to the model, instead of from the model config.
The logic here is intentionally designed to be easy to extend to support tasks across multiple runners if we ever decide to implement that.
Note: Since we didn't implement this
get_supported_tasksin the model runner of V0, this change only affects V1.cc @maxdebayser @noooop
Test Plan
Test Result
(Optional) Documentation Update