Skip to content

[Model] Use merge_by_field_config for MM models (D-F) #26076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 2, 2025
Merged

[Model] Use merge_by_field_config for MM models (D-F) #26076

merged 3 commits into from
Oct 2, 2025

Conversation

@DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented Oct 2, 2025 •
edited by github-actions bot
Loading

Purpose

Part of #26149

Test Plan

Model and tensor schema tests should pass. I have also run the example script on all four models.

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@DarkLight1337 DarkLight1337 requested a review from ywang96 October 2, 2025 04:47
@mergify mergify bot added the deepseek Related to DeepSeek models label Oct 2, 2025
@DarkLight1337
[Model] Use merge_by_field_config for MM models (D)
cf7c22d 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337 DarkLight1337 force-pushed the merge-by-field-config-2 branch from 6fbc76e to cf7c22d Compare October 2, 2025 04:48
@DarkLight1337 DarkLight1337 requested a review from Isotr0py October 2, 2025 04:49
gemini-code-assist[bot]
gemini-code-assist bot reviewed Oct 2, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors several multimodal models to use the new merge_by_field_config feature. This simplifies the input processing logic by removing manual tensor flattening code, such as flatten_bn and other custom reshaping functions. The changes are applied consistently across multiple models including Aria, Aya-Vision, Blip2, Chameleon, Cohere2-Vision, Deepseek-VL2, and Dots-OCR. The refactoring improves code clarity and maintainability by centralizing the input batching logic. The changes appear correct and well-executed.

DarkLight1337
DarkLight1337 commented Oct 2, 2025
View reviewed changes
Copy link
Member Author

@DarkLight1337 DarkLight1337 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/gemini review

@Isotr0py Isotr0py enabled auto-merge (squash) October 2, 2025 06:45
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 2, 2025
@DarkLight1337
Update Ernie and Fuyu as well
2a3ce6b 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337 DarkLight1337 changed the title [Model] Use merge_by_field_config for MM models (D) [Model] Use merge_by_field_config for MM models (D-F) Oct 2, 2025
@DarkLight1337
Copy link
Member Author

DarkLight1337 commented Oct 2, 2025

Updated Ernie4.5-VL and Fuyu as well

@vllm-bot vllm-bot merged commit cc253b7 into vllm-project:main Oct 2, 2025
49 of 53 checks passed
@DarkLight1337 DarkLight1337 deleted the merge-by-field-config-2 branch October 2, 2025 15:18
pdasigi pushed a commit to pdasigi/vllm that referenced this pull request Oct 2, 2025
@DarkLight1337 @pdasigi
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
ae28032 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337 DarkLight1337 mentioned this pull request Oct 3, 2025
Closed
13 tasks
yewentao256 pushed a commit that referenced this pull request Oct 3, 2025
@DarkLight1337 @yewentao256
[Model] Use merge_by_field_config for MM models (D-F) (#26076)
3884dce 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
tomeras91 pushed a commit to tomeras91/vllm that referenced this pull request Oct 6, 2025
@DarkLight1337 @tomeras91
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
635f277 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
southfreebird pushed a commit to southfreebird/vllm that referenced this pull request Oct 7, 2025
@DarkLight1337 @southfreebird
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
1e748bd 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
@DarkLight1337 @xuebwang-amd
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
90fa98f 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@DarkLight1337 @lywa1998
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
13d275a 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
@DarkLight1337 @alhridoy
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
7766cf6 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
@DarkLight1337 @xuebwang-amd
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
01ef916 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
@DarkLight1337 @rtourgeman
[Model] Use merge_by_field_config for MM models (D-F) (vllm-project…
efb8120 
…#26076)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Isotr0py Isotr0py Isotr0py approved these changes

@ywang96 ywang96 Awaiting requested review from ywang96

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

deepseek Related to DeepSeek models ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DarkLight1337 @Isotr0py @vllm-bot