Skip to content

Conversation

@DarkLight1337
Copy link
Member

@DarkLight1337 DarkLight1337 commented Oct 4, 2025

Purpose

Part of #26149

Test Plan

Tested all models other than Kimi-VL (doesn't work with latest Transformers)

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337 DarkLight1337 requested a review from Isotr0py October 4, 2025 14:56
@DarkLight1337 DarkLight1337 added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 4, 2025
@mergify mergify bot added the documentation Improvements or additions to documentation label Oct 4, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors several multi-modal models (Idefics3, Keye, Keye-VL 1.5, Kimi-VL) to use a new centralized mechanism for handling multi-modal inputs, enabled by the merge_by_field_config flag. This is a good maintainability improvement as it removes duplicated validation and reshaping code from individual model files. However, I've found a couple of potential issues. For Kimi-VL, specific reshaping logic has been removed, which is likely to break the model since it could not be tested. For Idefics3, enabling merge_by_field_config will use an existing configuration function that appears to be incorrect for image_embeds and num_patches. I've left detailed comments on these issues. The changes for other models and the improvements to error messages in tensor_schema.py are positive.

@DarkLight1337 DarkLight1337 requested a review from ywang96 October 4, 2025 16:19
@DarkLight1337
Copy link
Member Author

cc @Isotr0py

@Isotr0py Isotr0py merged commit 59a85c3 into vllm-project:main Oct 5, 2025
57 of 58 checks passed
@DarkLight1337 DarkLight1337 deleted the merge-by-field-config-4 branch October 5, 2025 04:02
tomeras91 pushed a commit to tomeras91/vllm that referenced this pull request Oct 6, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
karan pushed a commit to karan/vllm that referenced this pull request Oct 6, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>
southfreebird pushed a commit to southfreebird/vllm that referenced this pull request Oct 7, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
…#26230)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants