Skip to content

[Optimization] Streamline InputPreprocessor #25702

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 25, 2025
Merged

[Optimization] Streamline InputPreprocessor #25702

merged 1 commit into from
Sep 25, 2025

Conversation

@DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented Sep 25, 2025 •
edited by github-actions bot
Loading

Purpose

  • Initialize multimodal processor only once
  • Remove all async methods because they aren't being used in V1

FIX #25671

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@DarkLight1337
[Optimization] Streamline InputPreprocessor
5ee5ed5 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 25, 2025
@DarkLight1337 DarkLight1337 mentioned this pull request Sep 25, 2025
Closed
gemini-code-assist[bot]
gemini-code-assist bot reviewed Sep 25, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request streamlines the InputPreprocessor by caching the multimodal processor and removing unused async methods. The change to initialize the processor only once is a good optimization. However, I've identified a potential race condition in the lazy initialization logic that could occur in a multi-threaded environment. My review comment provides details on this issue and suggests a fix to ensure thread safety. The removal of async methods simplifies the codebase as intended.

@DarkLight1337 DarkLight1337 added this to the v0.11.0 milestone Sep 25, 2025
ProExpertProg
ProExpertProg reviewed Sep 25, 2025
View reviewed changes
Copy link
Collaborator

@ProExpertProg ProExpertProg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice cleanup. I'm not familiar enough to give a proper approve but I am very happy we're not hacking around on the ModelConfig hash!

@DarkLight1337 DarkLight1337 mentioned this pull request Sep 25, 2025
Merged
5 tasks
ywang96
ywang96 approved these changes Sep 25, 2025
View reviewed changes
Copy link
Member

@ywang96 ywang96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@DarkLight1337 DarkLight1337 enabled auto-merge (squash) September 25, 2025 19:16
@DarkLight1337 DarkLight1337 merged commit 3d54bdc into vllm-project:main Sep 25, 2025
44 of 45 checks passed
@DarkLight1337 DarkLight1337 deleted the remove-input-preprocessor-async branch September 25, 2025 21:06
yewentao256 pushed a commit that referenced this pull request Oct 3, 2025
@DarkLight1337 @yewentao256
[Optimization] Streamline InputPreprocessor (#25702)
745b204 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
@DarkLight1337 @xuebwang-amd
[Optimization] Streamline InputPreprocessor (vllm-project#25702)
6062817 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
@DarkLight1337 @choprahetarth
[Optimization] Streamline InputPreprocessor (vllm-project#25702)
b3e5f21 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@DarkLight1337 @lywa1998
[Optimization] Streamline InputPreprocessor (vllm-project#25702)
ca7b4eb 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
@DarkLight1337 @xuebwang-amd
[Optimization] Streamline InputPreprocessor (vllm-project#25702)
1ae78fa 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ProExpertProg ProExpertProg ProExpertProg left review comments

@ywang96 ywang96 ywang96 approved these changes

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Milestone

v0.11.0

Development

Successfully merging this pull request may close these issues.

[Performance] model_config.compute_hash is computed every time and introduce overhead in each new multi-modal req

3 participants

@DarkLight1337 @ProExpertProg @ywang96