-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
[Bugfix] Fix Qwen3-VL-MoE weight loading for EP #25300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request fixes a bug in the weight loading logic for Qwen3-VL-MoE models when using expert parallelism (EP). The previous implementation of load_fused_expert_weights was too strict, causing weight loading to fail if any expert was not present on a given rank. The change correctly modifies the logic to consider the loading successful if at least one expert's weights are loaded on the rank, which is the expected behavior for expert parallelism. The changes are correct and effectively address the issue. There is also a minor whitespace change to add a newline at the end of a file.
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io> Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Roger Wang <hey@rogerw.io> Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Roger Wang <hey@rogerw.io> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Roger Wang <hey@rogerw.io> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Purpose
Previously
self.load_fused_expert_weightscheck is too strict and will prevent server from launching with--enable-expert-parallel. This PR fixes it.Test Plan
The MMMU from server launched with/without
--enable-expert-parallelmatched.Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.