[Bug] Dynamo Unsupported due to BasevLLMParameter.torch_function calling disabled super()
#25613
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: yewentao256 <zhyanwentao@126.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a workaround for an issue with TorchDynamo in PyTorch versions older than 2.8.0. The change modifies BasevLLMParameter.__torch_function__ to return NotImplemented for these older versions, preventing a crash. The implementation is correct, but I have one suggestion to improve the logging behavior to avoid potential performance issues and log spam.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
mgoin
approved these changes
Sep 24, 2025
github-actions
bot
added
the
ready
|
@mgoin @yewentao256 This pr cause the deepseek model crash while loading: The output of tracebackCould it be further checked or tested? (vllm-metax plugin with torch 2.6 got crashed on e2e test, without the modification everything worked well) |
|
+1, hit the similar issue. |
|
@ILikeIneine Thanks for the feedback! Taking a look now |
|
Hey @ILikeIneine @wangxiyuan, is there a reason for your hardware backends to still be using torch 2.6 and 2.7? Generally we expect vllm to be compatible with only the latest version of torch and we don't have a need to support older torch versions before. If we must consider this, then we need to change our policy and CI to help capture issues in upstream. If I understand your comments correctly, you only have an issue with this "bugfix" PR right? We can revert it as it only tries to give a helpful log |
xuechendi
added a commit
to vllm-project/vllm-gaudi
that referenced
this pull request
Sep 25, 2025
vllm-project/vllm#23991 vllm-project/vllm#25613 --------- Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>
@mgoin Sorry for the delayed response. This is because our hardware backend sometimes can't catch up the latest release of torch. It takes some time to adapt. So the plugin torch version may not always the same as vllm's.(sometimes it's one or two version behind) |
|
yes, we use torch 2.7 now. |
iboiko-habana
pushed a commit
to iboiko-habana/vllm-gaudi
that referenced
this pull request
Oct 2, 2025
vllm-project/vllm#23991 vllm-project/vllm#25613 --------- Signed-off-by: Chendi Xue <Chendi.Xue@intel.com> Signed-off-by: Iryna Boiko <iboiko@habana.ai>
yewentao256
added a commit
that referenced
this pull request
Oct 3, 2025
…lling disabled super() (#25613) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: yewentao256 <zhyanwentao@126.com>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 10, 2025
…lling disabled super() (vllm-project#25613) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
choprahetarth
pushed a commit
to Tandemn-Labs/vllm
that referenced
this pull request
Oct 11, 2025
…lling disabled super() (vllm-project#25613) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
lywa1998
pushed a commit
to lywa1998/vllm
that referenced
this pull request
Oct 20, 2025
…lling disabled super() (vllm-project#25613) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 24, 2025
…lling disabled super() (vllm-project#25613) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Added clear instructions for #25604 (comment)
Test
After upgrading to torch 2.8.0, the issue disappear
vllm bench throughput --model Qwen/Qwen3-30B-A3B-FP8 --load-format dummy --input-len 1000 --output-len 100 --trust_remote_code --enable-expert-parallel