-
-
Notifications
You must be signed in to change notification settings - Fork 11.3k
[Model] Use merge_by_field_config for MM models (Qwen series) #27546
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Model] Use merge_by_field_config for MM models (Qwen series) #27546
Conversation
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request refactors the multi-modal input handling for Qwen series models by leveraging the merge_by_field_config mechanism. This is a good improvement as it centralizes the input processing logic, removes redundant code (like _validate_and_reshape_mm_tensor methods) from individual model files, and makes the codebase cleaner and more maintainable. The changes across qwen2_5_omni_thinker.py, qwen2_5_vl.py, qwen2_audio.py, qwen2_vl.py, qwen3_omni_moe_thinker.py, qwen3_vl.py, and qwen_vl.py are consistent with this goal.
I've found one issue that needs to be addressed before merging.
…roject#27546) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…roject#27546) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…roject#27546) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
…roject#27546) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Purpose
CLOSE #26149
Test Plan
The tensor schema tests passed locally (Qwen3 series isn't tested in CI yet). I have also run the example scripts for each model (except for Qwen3-Omni because it doesn't have one)
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.