-
Notifications
You must be signed in to change notification settings - Fork 533
[Core] Fix block table shape to make Prefix cache work with Ascend scheduler #1446
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1446 +/- ##
==========================================
+ Coverage 27.39% 27.41% +0.01%
==========================================
Files 56 56
Lines 6191 6271 +80
==========================================
+ Hits 1696 1719 +23
- Misses 4495 4552 +57
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
wangxiyuan
approved these changes
Jun 27, 2025
47 tasks
|
Add e2e test in #1505 |
wangxiyuan
approved these changes
Jun 30, 2025
weijinqian0
pushed a commit
to weijinqian0/vllm-ascend
that referenced
this pull request
Jun 30, 2025
…heduler (vllm-project#1446) ### What this PR does / why we need it? This fix the shape of block_table which was introduced by hybrid kv groups several weeks ago. Error will be raised when enable prefix-cache (eager or not) and Ascend Scheduler at the same time, just send two identical requests and it will reproduce. v0.9.1: vllm-project#1297 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Test manually Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
Yikun
pushed a commit
that referenced
this pull request
Jul 2, 2025
…uler (#1505) ### What this PR does / why we need it? Add test for chunked prefill and prefix cache on v1/AscendScheduler Covered scenarios: - `Qwen/Qwen3-0.6B-Base` and `deepseek-ai/DeepSeek-V2-Lite-Chat` --- multicard CI time increased by 19 min - `V1 + default scheduler` vs `V1 + default scheduler + enable prefix cache` - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable prefix cache` vs `V1 + Ascend scheduler + enable prefix cache + enable chunked prefill` - `Qwen/Qwen3-0.6B-Base` --- singlecard CI time increased by 8 min - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable chunked prefill` should rebase after #1498 and #1446 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? CI passed with new added test. Signed-off-by: MengqingCao <cmq0113@163.com>
ZhengWG
pushed a commit
to ZhengWG/vllm-ascend
that referenced
this pull request
Jul 3, 2025
…uler (vllm-project#1505) ### What this PR does / why we need it? Add test for chunked prefill and prefix cache on v1/AscendScheduler Covered scenarios: - `Qwen/Qwen3-0.6B-Base` and `deepseek-ai/DeepSeek-V2-Lite-Chat` --- multicard CI time increased by 19 min - `V1 + default scheduler` vs `V1 + default scheduler + enable prefix cache` - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable prefix cache` vs `V1 + Ascend scheduler + enable prefix cache + enable chunked prefill` - `Qwen/Qwen3-0.6B-Base` --- singlecard CI time increased by 8 min - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable chunked prefill` should rebase after vllm-project#1498 and vllm-project#1446 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? CI passed with new added test. Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: ZhengWG <zwg0606@gmail.com>
ZhengWG
pushed a commit
to ZhengWG/vllm-ascend
that referenced
this pull request
Jul 3, 2025
…uler (vllm-project#1505) ### What this PR does / why we need it? Add test for chunked prefill and prefix cache on v1/AscendScheduler Covered scenarios: - `Qwen/Qwen3-0.6B-Base` and `deepseek-ai/DeepSeek-V2-Lite-Chat` --- multicard CI time increased by 19 min - `V1 + default scheduler` vs `V1 + default scheduler + enable prefix cache` - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable prefix cache` vs `V1 + Ascend scheduler + enable prefix cache + enable chunked prefill` - `Qwen/Qwen3-0.6B-Base` --- singlecard CI time increased by 8 min - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable chunked prefill` should rebase after vllm-project#1498 and vllm-project#1446 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? CI passed with new added test. Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: ZhengWG <zwg0606@gmail.com>
Yikun
added a commit
that referenced
this pull request
Jul 6, 2025
### What this PR does / why we need it? Fix the shape of the `npu_moe_init_routing` input parameters to support aclgraph mode on qwen3-moe In addition to this PR, resolving the `gatherv3` error might be necessary. See related PR #1297 #1446 Thanks to @yiz-liu for providing the idea ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested on Qwen3-30B-A3B Closes: #1368 --------- Signed-off-by: ApsarasX <apsarax@outlook.com> Signed-off-by: Yikun Jiang <yikunkero@gmail.com> Co-authored-by: Yizhou Liu <liu_yizhou@outlook.com> Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
This was referenced Jul 13, 2025
MengqingCao
pushed a commit
that referenced
this pull request
Aug 19, 2025
I would like to nominate Mengqing Cao (@MengqingCao https://github.com/MengqingCao) as a maintainer, starting with my +1. ## Reason Review Quality: She has completed [120+ reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao) since Feb. 2025, include [#review-3077842852](#2088 (review)), [comment-2990074116](#1032 (comment)), [comment-2921063723](#1013 (comment)) high quality review. Sustained and Quality Contributions: She has Deep understanding of vLLM and vLLM Ascend codebases and solid contributions include The vLLM contributions and help vLLM Ascend release is the main reason I nominated her: - vLLM: Things worth mentioning that she completed [28+ PR contributions](https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+) in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support. She is one of the important co-authors of [vllm#8054](vllm-project/vllm#8054) and hardware plugin RFC, this makes vllm-ascend plugin possible. Community Involvement: She is also very active and involved in [60+ issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao). So I think she's a great addition to the vLLM Ascend Maintainer team. - ✅**Review Quality:** She has completed 120+ reviews since Feb. 2025. https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao, include #2088 (review), #1446 (comment), #1032 (comment), #1013 (comment) quality review. - ✅**Sustained Contributions:** 99+ PR merged in vllm-project/vllm-ascend https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged - ✅**Quality Contribution:** She is one of the important co-authors of vllm-project/vllm#8054 , this makes vllm-ascend plugin possible. Things worth mentioning that she complete 28+ PR contributions in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support: https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+. At 2025 Q2, She also lead the [[RFC]: E2E CI test for key features](#413) and [[RFC]: Unit test coverage improvement](#1298) to help vllm ascend improve the coverage. Her main contributions focus on the adaptation of parallel strategies and communicator, such as #1800, #1856. These contributions are sufficient to prove she has “Deep understanding of vLLM and vLLM Ascend codebases” - ✅**Community Involvement:** Involved in 63+ issue reviewer https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao She led the v0.10.1 release as release manager - vLLM version: v0.10.0 - vLLM main: vllm-project/vllm@78dba40 Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
wangxiaoteng888
pushed a commit
to LCAIZJ/vllm-ascend
that referenced
this pull request
Sep 25, 2025
I would like to nominate Mengqing Cao (@MengqingCao https://github.com/MengqingCao) as a maintainer, starting with my +1. ## Reason Review Quality: She has completed [120+ reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao) since Feb. 2025, include [#review-3077842852](vllm-project#2088 (review)), [comment-2990074116](vllm-project#1032 (comment)), [comment-2921063723](vllm-project#1013 (comment)) high quality review. Sustained and Quality Contributions: She has Deep understanding of vLLM and vLLM Ascend codebases and solid contributions include The vLLM contributions and help vLLM Ascend release is the main reason I nominated her: - vLLM: Things worth mentioning that she completed [28+ PR contributions](https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+) in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support. She is one of the important co-authors of [vllm#8054](vllm-project/vllm#8054) and hardware plugin RFC, this makes vllm-ascend plugin possible. Community Involvement: She is also very active and involved in [60+ issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao). So I think she's a great addition to the vLLM Ascend Maintainer team. - ✅**Review Quality:** She has completed 120+ reviews since Feb. 2025. https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao, include vllm-project#2088 (review), vllm-project#1446 (comment), vllm-project#1032 (comment), vllm-project#1013 (comment) quality review. - ✅**Sustained Contributions:** 99+ PR merged in vllm-project/vllm-ascend https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged - ✅**Quality Contribution:** She is one of the important co-authors of vllm-project/vllm#8054 , this makes vllm-ascend plugin possible. Things worth mentioning that she complete 28+ PR contributions in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support: https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+. At 2025 Q2, She also lead the [[RFC]: E2E CI test for key features](vllm-project#413) and [[RFC]: Unit test coverage improvement](vllm-project#1298) to help vllm ascend improve the coverage. Her main contributions focus on the adaptation of parallel strategies and communicator, such as vllm-project#1800, vllm-project#1856. These contributions are sufficient to prove she has “Deep understanding of vLLM and vLLM Ascend codebases” - ✅**Community Involvement:** Involved in 63+ issue reviewer https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao She led the v0.10.1 release as release manager - vLLM version: v0.10.0 - vLLM main: vllm-project/vllm@78dba40 Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Sep 26, 2025
I would like to nominate Mengqing Cao (@MengqingCao https://github.com/MengqingCao) as a maintainer, starting with my +1. ## Reason Review Quality: She has completed [120+ reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao) since Feb. 2025, include [#review-3077842852](vllm-project#2088 (review)), [comment-2990074116](vllm-project#1032 (comment)), [comment-2921063723](vllm-project#1013 (comment)) high quality review. Sustained and Quality Contributions: She has Deep understanding of vLLM and vLLM Ascend codebases and solid contributions include The vLLM contributions and help vLLM Ascend release is the main reason I nominated her: - vLLM: Things worth mentioning that she completed [28+ PR contributions](https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+) in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support. She is one of the important co-authors of [vllm#8054](vllm-project/vllm#8054) and hardware plugin RFC, this makes vllm-ascend plugin possible. Community Involvement: She is also very active and involved in [60+ issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao). So I think she's a great addition to the vLLM Ascend Maintainer team. - ✅**Review Quality:** She has completed 120+ reviews since Feb. 2025. https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao, include vllm-project#2088 (review), vllm-project#1446 (comment), vllm-project#1032 (comment), vllm-project#1013 (comment) quality review. - ✅**Sustained Contributions:** 99+ PR merged in vllm-project/vllm-ascend https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged - ✅**Quality Contribution:** She is one of the important co-authors of vllm-project/vllm#8054 , this makes vllm-ascend plugin possible. Things worth mentioning that she complete 28+ PR contributions in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support: https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+. At 2025 Q2, She also lead the [[RFC]: E2E CI test for key features](vllm-project#413) and [[RFC]: Unit test coverage improvement](vllm-project#1298) to help vllm ascend improve the coverage. Her main contributions focus on the adaptation of parallel strategies and communicator, such as vllm-project#1800, vllm-project#1856. These contributions are sufficient to prove she has “Deep understanding of vLLM and vLLM Ascend codebases” - ✅**Community Involvement:** Involved in 63+ issue reviewer https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao She led the v0.10.1 release as release manager - vLLM version: v0.10.0 - vLLM main: vllm-project/vllm@78dba40 Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Oct 16, 2025
…heduler (vllm-project#1446) ### What this PR does / why we need it? This fix the shape of block_table which was introduced by hybrid kv groups several weeks ago. Error will be raised when enable prefix-cache (eager or not) and Ascend Scheduler at the same time, just send two identical requests and it will reproduce. v0.9.1: vllm-project#1297 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Test manually Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Oct 16, 2025
…uler (vllm-project#1505) ### What this PR does / why we need it? Add test for chunked prefill and prefix cache on v1/AscendScheduler Covered scenarios: - `Qwen/Qwen3-0.6B-Base` and `deepseek-ai/DeepSeek-V2-Lite-Chat` --- multicard CI time increased by 19 min - `V1 + default scheduler` vs `V1 + default scheduler + enable prefix cache` - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable prefix cache` vs `V1 + Ascend scheduler + enable prefix cache + enable chunked prefill` - `Qwen/Qwen3-0.6B-Base` --- singlecard CI time increased by 8 min - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable chunked prefill` should rebase after vllm-project#1498 and vllm-project#1446 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? CI passed with new added test. Signed-off-by: MengqingCao <cmq0113@163.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Oct 16, 2025
### What this PR does / why we need it? Fix the shape of the `npu_moe_init_routing` input parameters to support aclgraph mode on qwen3-moe In addition to this PR, resolving the `gatherv3` error might be necessary. See related PR vllm-project#1297 vllm-project#1446 Thanks to @yiz-liu for providing the idea ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested on Qwen3-30B-A3B Closes: vllm-project#1368 --------- Signed-off-by: ApsarasX <apsarax@outlook.com> Signed-off-by: Yikun Jiang <yikunkero@gmail.com> Co-authored-by: Yizhou Liu <liu_yizhou@outlook.com> Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
…heduler (vllm-project#1446) ### What this PR does / why we need it? This fix the shape of block_table which was introduced by hybrid kv groups several weeks ago. Error will be raised when enable prefix-cache (eager or not) and Ascend Scheduler at the same time, just send two identical requests and it will reproduce. v0.9.1: vllm-project#1297 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Test manually Signed-off-by: Yizhou Liu <liu_yizhou@outlook.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
…uler (vllm-project#1505) ### What this PR does / why we need it? Add test for chunked prefill and prefix cache on v1/AscendScheduler Covered scenarios: - `Qwen/Qwen3-0.6B-Base` and `deepseek-ai/DeepSeek-V2-Lite-Chat` --- multicard CI time increased by 19 min - `V1 + default scheduler` vs `V1 + default scheduler + enable prefix cache` - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable prefix cache` vs `V1 + Ascend scheduler + enable prefix cache + enable chunked prefill` - `Qwen/Qwen3-0.6B-Base` --- singlecard CI time increased by 8 min - `V1 + Ascend scheduler` vs `V1 + Ascend scheduler + enable chunked prefill` should rebase after vllm-project#1498 and vllm-project#1446 ### Does this PR introduce _any_ user-facing change? N/A ### How was this patch tested? CI passed with new added test. Signed-off-by: MengqingCao <cmq0113@163.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
### What this PR does / why we need it? Fix the shape of the `npu_moe_init_routing` input parameters to support aclgraph mode on qwen3-moe In addition to this PR, resolving the `gatherv3` error might be necessary. See related PR vllm-project#1297 vllm-project#1446 Thanks to @yiz-liu for providing the idea ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested on Qwen3-30B-A3B Closes: vllm-project#1368 --------- Signed-off-by: ApsarasX <apsarax@outlook.com> Signed-off-by: Yikun Jiang <yikunkero@gmail.com> Co-authored-by: Yizhou Liu <liu_yizhou@outlook.com> Co-authored-by: Yikun Jiang <yikunkero@gmail.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
I would like to nominate Mengqing Cao (@MengqingCao https://github.com/MengqingCao) as a maintainer, starting with my +1. ## Reason Review Quality: She has completed [120+ reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao) since Feb. 2025, include [#review-3077842852](vllm-project#2088 (review)), [comment-2990074116](vllm-project#1032 (comment)), [comment-2921063723](vllm-project#1013 (comment)) high quality review. Sustained and Quality Contributions: She has Deep understanding of vLLM and vLLM Ascend codebases and solid contributions include The vLLM contributions and help vLLM Ascend release is the main reason I nominated her: - vLLM: Things worth mentioning that she completed [28+ PR contributions](https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+) in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support. She is one of the important co-authors of [vllm#8054](vllm-project/vllm#8054) and hardware plugin RFC, this makes vllm-ascend plugin possible. Community Involvement: She is also very active and involved in [60+ issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao). So I think she's a great addition to the vLLM Ascend Maintainer team. - ✅**Review Quality:** She has completed 120+ reviews since Feb. 2025. https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+is%3Aclosed+commenter%3Amengqingcao+-author%3Amengqingcao, include vllm-project#2088 (review), vllm-project#1446 (comment), vllm-project#1032 (comment), vllm-project#1013 (comment) quality review. - ✅**Sustained Contributions:** 99+ PR merged in vllm-project/vllm-ascend https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged - ✅**Quality Contribution:** She is one of the important co-authors of vllm-project/vllm#8054 , this makes vllm-ascend plugin possible. Things worth mentioning that she complete 28+ PR contributions in vllm-project/vllm, especially for vLLM platform module to improve vLLM mult hardware support: https://github.com/vllm-project/vllm/pulls?q=is%3Apr+author%3AMengqingCao+is%3Amerged+. At 2025 Q2, She also lead the [[RFC]: E2E CI test for key features](vllm-project#413) and [[RFC]: Unit test coverage improvement](vllm-project#1298) to help vllm ascend improve the coverage. Her main contributions focus on the adaptation of parallel strategies and communicator, such as vllm-project#1800, vllm-project#1856. These contributions are sufficient to prove she has “Deep understanding of vLLM and vLLM Ascend codebases” - ✅**Community Involvement:** Involved in 63+ issue reviewer https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20-author%3AMengqingCao%20commenter%3AMengqingCao She led the v0.10.1 release as release manager - vLLM version: v0.10.0 - vLLM main: vllm-project/vllm@78dba40 Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
This fix the shape of block_table which was introduced by hybrid kv groups several weeks ago.
Error will be raised when enable prefix-cache (eager or not) and Ascend Scheduler at the same time, just send two identical requests and it will reproduce.
v0.9.1: #1297
Does this PR introduce any user-facing change?
No
How was this patch tested?
Test manually