Skip to content

[Renderer] Move Processor out of LLMEngine #26165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 3, 2025
Merged

[Renderer] Move Processor out of LLMEngine #26165

merged 3 commits into from
Oct 3, 2025

Conversation

@DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented Oct 3, 2025 •
edited by github-actions bot
Loading

Purpose

Similar to #24138, but concerning the LLMEngine for offline inference

Close #23869

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@DarkLight1337
[Renderer] Move Processor out of LLMEngine
a7166d0 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 3, 2025
gemini-code-assist[bot]
gemini-code-assist bot reviewed Oct 3, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the LLM class to move the Processor logic out of LLMEngine, which is a good step towards cleaner separation of concerns. The changes are mostly correct, but I found a critical issue in how the prompt text is handled after the refactoring. The prompt_text is not being passed correctly, which would lead to a runtime error. My review includes a fix for this issue.

@DarkLight1337
Address comment
c108543 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
Do it in another PR
2035cca 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@Isotr0py Isotr0py enabled auto-merge (squash) October 3, 2025 13:10
@Isotr0py Isotr0py merged commit d78fda7 into vllm-project:main Oct 3, 2025
50 checks passed
@DarkLight1337 DarkLight1337 deleted the processor-out-llm-engine branch October 3, 2025 15:51
yewentao256 pushed a commit that referenced this pull request Oct 3, 2025
@DarkLight1337 @yewentao256
[Renderer] Move Processor out of LLMEngine (#26165)
611c23b 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
MatthewBonanni pushed a commit to MatthewBonanni/vllm that referenced this pull request Oct 3, 2025
@DarkLight1337 @MatthewBonanni
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
85b4ce8 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
tomeras91 pushed a commit to tomeras91/vllm that referenced this pull request Oct 6, 2025
@DarkLight1337 @tomeras91
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
5a0dcdf 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
karan pushed a commit to karan/vllm that referenced this pull request Oct 6, 2025
@DarkLight1337 @karan
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
8af8422 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>
southfreebird pushed a commit to southfreebird/vllm that referenced this pull request Oct 7, 2025
@DarkLight1337 @southfreebird
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
1bc9aa6 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
@DarkLight1337 @xuebwang-amd
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
1353e1b 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@DarkLight1337 @lywa1998
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
9de037f 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
@DarkLight1337 @alhridoy
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
2630ea2 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
@DarkLight1337 @xuebwang-amd
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
8d824e2 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
@DarkLight1337 @rtourgeman
[Renderer] Move Processor out of LLMEngine (vllm-project#26165)
c919c65 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Isotr0py Isotr0py Isotr0py approved these changes

@ywang96 ywang96 Awaiting requested review from ywang96

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat

@njhill njhill Awaiting requested review from njhill

@comaniac comaniac Awaiting requested review from comaniac

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat

@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner

@chaunceyjiang chaunceyjiang Awaiting requested review from chaunceyjiang chaunceyjiang is a code owner

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

frontend ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Renderer]: Move Processor out of AsyncLLM and LLMEngine

2 participants

@DarkLight1337 @Isotr0py