[BugFix] Fix chunked prefill bugs in engine v1 #844
Merged
+14
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rjg-lyh
changed the title
rjg-lyh
force-pushed
the
pr-bugfix-dsv
branch
2 times, most recently
from
1e2aeea to
6725d90
Compare
rjg-lyh
force-pushed
the
pr-bugfix-dsv
branch
10 times, most recently
from
cb1df18 to
70dc428
Compare
Collaborator
diff --git a/vllm_ascend/patch/platform/patch_common/patch_vllm_config.py b/vllm_ascend/patch/platform/patch_common/patch_vllm_config.py
index 6d606d0..947ec7d 100644
--- a/vllm_ascend/patch/platform/patch_common/patch_vllm_config.py
+++ b/vllm_ascend/patch/platform/patch_common/patch_vllm_config.py
@@ -18,11 +18,10 @@
# This file is a part of the vllm-ascend project.
import torch
-
+import vllm.envs as envs
+from vllm.config import CompilationConfig, CompilationLevel, VllmConfig
from vllm.logger import init_logger
-from vllm.config import (VllmConfig, CompilationConfig, CompilationLevel)
from vllm.utils import random_uuid
-import vllm.envs as envs
logger = init_logger(__name__) |
rjg-lyh
force-pushed
the
pr-bugfix-dsv
branch
3 times, most recently
from
dfef60e to
e5e9548
Compare
Signed-off-by: rjg-lyh <1318825571@qq.com>
wangxiyuan
reviewed
May 21, 2025
| self.instance_id = random_uuid()[:5] | ||
|
|
||
|
|
||
| VllmConfig.__post_init__ = __post_init__ |
Collaborator
There was a problem hiding this comment.
patch vllm config is dengerous. it's always changed by vllm. How we make sure the capability?
Collaborator
There was a problem hiding this comment.
related PR to fix the issue: vllm-project/vllm#18470
Signed-off-by: rjg-lyh <1318825571@qq.com>
wangxiyuan
approved these changes
May 22, 2025
wangxiyuan
pushed a commit
that referenced
this pull request
May 30, 2025
### What this PR does / why we need it? add basic v1 mtp features please merge it after #874 and #844. ### Does this PR introduce _any_ user-facing change? now, we supported basic v1 mtp, only supported tp only、eager mode and k=1 we will continue to expand more scenarios. ### How was this patch tested? local tested Signed-off-by: XWFAlone <xuewenfei2@huawei.com> Co-authored-by: mengwei805 <mengwei25@huawei.com> Co-authored-by: JC-ut0 <xuyexiong@huawei.com>
momo609
pushed a commit
to momo609/vllm-ascend
that referenced
this pull request
Jun 3, 2025
### What this PR does / why we need it? add basic v1 mtp features please merge it after vllm-project#874 and vllm-project#844. ### Does this PR introduce _any_ user-facing change? now, we supported basic v1 mtp, only supported tp only、eager mode and k=1 we will continue to expand more scenarios. ### How was this patch tested? local tested Signed-off-by: XWFAlone <xuewenfei2@huawei.com> Co-authored-by: mengwei805 <mengwei25@huawei.com> Co-authored-by: JC-ut0 <xuyexiong@huawei.com> Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
David9857
pushed a commit
to David9857/vllm-ascend
that referenced
this pull request
Jun 3, 2025
### What this PR does / why we need it? add basic v1 mtp features please merge it after vllm-project#874 and vllm-project#844. ### Does this PR introduce _any_ user-facing change? now, we supported basic v1 mtp, only supported tp only、eager mode and k=1 we will continue to expand more scenarios. ### How was this patch tested? local tested Signed-off-by: XWFAlone <xuewenfei2@huawei.com> Co-authored-by: mengwei805 <mengwei25@huawei.com> Co-authored-by: JC-ut0 <xuyexiong@huawei.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Oct 16, 2025
### What this PR does / why we need it? Fix the bugs when run deepseek model in engine v1. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI passed with new added/existing test. --------- Signed-off-by: rjg-lyh <1318825571@qq.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Oct 16, 2025
### What this PR does / why we need it? add basic v1 mtp features please merge it after vllm-project#874 and vllm-project#844. ### Does this PR introduce _any_ user-facing change? now, we supported basic v1 mtp, only supported tp only、eager mode and k=1 we will continue to expand more scenarios. ### How was this patch tested? local tested Signed-off-by: XWFAlone <xuewenfei2@huawei.com> Co-authored-by: mengwei805 <mengwei25@huawei.com> Co-authored-by: JC-ut0 <xuyexiong@huawei.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
### What this PR does / why we need it? Fix the bugs when run deepseek model in engine v1. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI passed with new added/existing test. --------- Signed-off-by: rjg-lyh <1318825571@qq.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
### What this PR does / why we need it? add basic v1 mtp features please merge it after vllm-project#874 and vllm-project#844. ### Does this PR introduce _any_ user-facing change? now, we supported basic v1 mtp, only supported tp only、eager mode and k=1 we will continue to expand more scenarios. ### How was this patch tested? local tested Signed-off-by: XWFAlone <xuewenfei2@huawei.com> Co-authored-by: mengwei805 <mengwei25@huawei.com> Co-authored-by: JC-ut0 <xuyexiong@huawei.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
Fix the bugs when run deepseek model in engine v1.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
CI passed with new added/existing test.