[Bugfix] fix accuracy problem for quantized deepseek models #768
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ApsarasX
reviewed
May 6, 2025
linfeng-yuan
force-pushed
the
fix_acc
branch
5 times, most recently
from
0ea7d83 to
780f5d1
Compare
Signed-off-by: linfeng-yuan <1102311262@qq.com>
wangxiyuan
approved these changes
May 6, 2025
ApsarasX
pushed a commit
that referenced
this pull request
Aug 19, 2025
I would like to nominate Wengang Chen (@ApsarasX https://github.com/ApsarasX) as a maintainer, starting with my +1. ## Reason Review Quality: He focuses on the vLLM Ascend Core module review with 100+ high quality review, such as [#2326 (comment)](#2326 (comment)), [#768 (comment)](#768 (comment)), [#2312 (comment)](#2312 (comment)), [#2268 (comment)](#2268 (comment)), [#2192 (comment)](#2192 (comment)), [#2156 (comment)](#2156 (comment)). This helped vLLM Ascend v0.9.x and v0.10.x to be released with high quality. Sustained and Quality Contributions: He has a very good habit of sharing his design ideas, development process, performance test results, such as [#966](#966), he contributed [many PRs](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Amerged+), valuable bugfixes and also perf improvements. Community Involvement: Active involved in community discussion, he is collaborative and helps the users solve problems, involved in [120+ PR and issues](https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX). He is also the speaker of [vLLM Beijing Meetup](https://mp.weixin.qq.com/s/7n8OYNrCC_I9SJaybHA_-Q). So I think he's a great addition to the vLLM Ascend Maintainer team. - ✅Review Quality: 108+ PR with valuable review https://github.com/vllm-project/vllm-ascend/pulls?q=commenter%3AApsarasX with many valuable review, like #2326 (comment) #768 (comment) #2312 (comment) #2268 (comment) #2192 (comment) #2156 (comment) - ✅ Sustained and Major Contributions https://github.com/vllm-project/vllm-ascend/pulls/ApsarasX - ✅ Quality Contribution: https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Aclosed Good quality with well documents [Perf] Refactor tensor disposal logic to reduce memory usage #966 - ✅Community Involvement: 7 issue: https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20author%3AApsarasX - 120+ PR and issue: https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
wangxiaoteng888
pushed a commit
to LCAIZJ/vllm-ascend
that referenced
this pull request
Sep 25, 2025
I would like to nominate Wengang Chen (@ApsarasX https://github.com/ApsarasX) as a maintainer, starting with my +1. ## Reason Review Quality: He focuses on the vLLM Ascend Core module review with 100+ high quality review, such as [vllm-project#2326 (comment)](vllm-project#2326 (comment)), [vllm-project#768 (comment)](vllm-project#768 (comment)), [vllm-project#2312 (comment)](vllm-project#2312 (comment)), [vllm-project#2268 (comment)](vllm-project#2268 (comment)), [vllm-project#2192 (comment)](vllm-project#2192 (comment)), [vllm-project#2156 (comment)](vllm-project#2156 (comment)). This helped vLLM Ascend v0.9.x and v0.10.x to be released with high quality. Sustained and Quality Contributions: He has a very good habit of sharing his design ideas, development process, performance test results, such as [vllm-project#966](vllm-project#966), he contributed [many PRs](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Amerged+), valuable bugfixes and also perf improvements. Community Involvement: Active involved in community discussion, he is collaborative and helps the users solve problems, involved in [120+ PR and issues](https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX). He is also the speaker of [vLLM Beijing Meetup](https://mp.weixin.qq.com/s/7n8OYNrCC_I9SJaybHA_-Q). So I think he's a great addition to the vLLM Ascend Maintainer team. - ✅Review Quality: 108+ PR with valuable review https://github.com/vllm-project/vllm-ascend/pulls?q=commenter%3AApsarasX with many valuable review, like vllm-project#2326 (comment) vllm-project#768 (comment) vllm-project#2312 (comment) vllm-project#2268 (comment) vllm-project#2192 (comment) vllm-project#2156 (comment) - ✅ Sustained and Major Contributions https://github.com/vllm-project/vllm-ascend/pulls/ApsarasX - ✅ Quality Contribution: https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Aclosed Good quality with well documents [Perf] Refactor tensor disposal logic to reduce memory usage vllm-project#966 - ✅Community Involvement: 7 issue: https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20author%3AApsarasX - 120+ PR and issue: https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Sep 26, 2025
I would like to nominate Wengang Chen (@ApsarasX https://github.com/ApsarasX) as a maintainer, starting with my +1. ## Reason Review Quality: He focuses on the vLLM Ascend Core module review with 100+ high quality review, such as [vllm-project#2326 (comment)](vllm-project#2326 (comment)), [vllm-project#768 (comment)](vllm-project#768 (comment)), [vllm-project#2312 (comment)](vllm-project#2312 (comment)), [vllm-project#2268 (comment)](vllm-project#2268 (comment)), [vllm-project#2192 (comment)](vllm-project#2192 (comment)), [vllm-project#2156 (comment)](vllm-project#2156 (comment)). This helped vLLM Ascend v0.9.x and v0.10.x to be released with high quality. Sustained and Quality Contributions: He has a very good habit of sharing his design ideas, development process, performance test results, such as [vllm-project#966](vllm-project#966), he contributed [many PRs](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Amerged+), valuable bugfixes and also perf improvements. Community Involvement: Active involved in community discussion, he is collaborative and helps the users solve problems, involved in [120+ PR and issues](https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX). He is also the speaker of [vLLM Beijing Meetup](https://mp.weixin.qq.com/s/7n8OYNrCC_I9SJaybHA_-Q). So I think he's a great addition to the vLLM Ascend Maintainer team. - ✅Review Quality: 108+ PR with valuable review https://github.com/vllm-project/vllm-ascend/pulls?q=commenter%3AApsarasX with many valuable review, like vllm-project#2326 (comment) vllm-project#768 (comment) vllm-project#2312 (comment) vllm-project#2268 (comment) vllm-project#2192 (comment) vllm-project#2156 (comment) - ✅ Sustained and Major Contributions https://github.com/vllm-project/vllm-ascend/pulls/ApsarasX - ✅ Quality Contribution: https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Aclosed Good quality with well documents [Perf] Refactor tensor disposal logic to reduce memory usage vllm-project#966 - ✅Community Involvement: 7 issue: https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20author%3AApsarasX - 120+ PR and issue: https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Oct 16, 2025
…ject#768) ### What this PR does / why we need it? The root cause of the bug is that numerical computations involving NaNs cannot eliminate them. We addressed it by using `masked_fill_` to eliminate NaNs while avoiding memory-wasting `torch.where` approach. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? This patch was tested with vllm v0.8.5 and vllm-ascend master. I run deepseek_v3 model with offline inference scripts (examples/dp_offline/run_dp.sh & data_parallel.py). Signed-off-by: linfeng-yuan <1102311262@qq.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
…ject#768) ### What this PR does / why we need it? The root cause of the bug is that numerical computations involving NaNs cannot eliminate them. We addressed it by using `masked_fill_` to eliminate NaNs while avoiding memory-wasting `torch.where` approach. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? This patch was tested with vllm v0.8.5 and vllm-ascend master. I run deepseek_v3 model with offline inference scripts (examples/dp_offline/run_dp.sh & data_parallel.py). Signed-off-by: linfeng-yuan <1102311262@qq.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
I would like to nominate Wengang Chen (@ApsarasX https://github.com/ApsarasX) as a maintainer, starting with my +1. ## Reason Review Quality: He focuses on the vLLM Ascend Core module review with 100+ high quality review, such as [vllm-project#2326 (comment)](vllm-project#2326 (comment)), [vllm-project#768 (comment)](vllm-project#768 (comment)), [vllm-project#2312 (comment)](vllm-project#2312 (comment)), [vllm-project#2268 (comment)](vllm-project#2268 (comment)), [vllm-project#2192 (comment)](vllm-project#2192 (comment)), [vllm-project#2156 (comment)](vllm-project#2156 (comment)). This helped vLLM Ascend v0.9.x and v0.10.x to be released with high quality. Sustained and Quality Contributions: He has a very good habit of sharing his design ideas, development process, performance test results, such as [vllm-project#966](vllm-project#966), he contributed [many PRs](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Amerged+), valuable bugfixes and also perf improvements. Community Involvement: Active involved in community discussion, he is collaborative and helps the users solve problems, involved in [120+ PR and issues](https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX). He is also the speaker of [vLLM Beijing Meetup](https://mp.weixin.qq.com/s/7n8OYNrCC_I9SJaybHA_-Q). So I think he's a great addition to the vLLM Ascend Maintainer team. - ✅Review Quality: 108+ PR with valuable review https://github.com/vllm-project/vllm-ascend/pulls?q=commenter%3AApsarasX with many valuable review, like vllm-project#2326 (comment) vllm-project#768 (comment) vllm-project#2312 (comment) vllm-project#2268 (comment) vllm-project#2192 (comment) vllm-project#2156 (comment) - ✅ Sustained and Major Contributions https://github.com/vllm-project/vllm-ascend/pulls/ApsarasX - ✅ Quality Contribution: https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3AApsarasX+is%3Aclosed Good quality with well documents [Perf] Refactor tensor disposal logic to reduce memory usage vllm-project#966 - ✅Community Involvement: 7 issue: https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aclosed%20author%3AApsarasX - 120+ PR and issue: https://github.com/vllm-project/vllm-ascend/issues?q=commenter%3AApsarasX Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
The root cause of the bug is that numerical computations involving NaNs cannot eliminate them. We addressed it by using
masked_fill_to eliminate NaNs while avoiding memory-wastingtorch.whereapproach.Does this PR introduce any user-facing change?
Not invovled.
How was this patch tested?
This patch was tested with vllm v0.8.5 and vllm-ascend master. I run deepseek_v3 model with offline inference scripts (examples/dp_offline/run_dp.sh & data_parallel.py).