Skip to content

[Bugfix] Fix deepseek percision issue and add acc ci for it #905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 4, 2025
Merged

[Bugfix] Fix deepseek percision issue and add acc ci for it #905

merged 1 commit into from
Jun 4, 2025

Conversation

@MengqingCao
Copy link
Collaborator

@MengqingCao MengqingCao commented May 20, 2025 •
edited
Loading

What this PR does / why we need it?

Fix deepseek percision issue on V0 and add acc ci for it
Fixes #1062

How was this patch tested?

CI passed with new added test.

@MengqingCao MengqingCao marked this pull request as ready for review May 20, 2025 06:29
MengqingCao
MengqingCao commented May 20, 2025
View reviewed changes
vllm_ascend/ops/fused_moe.py Outdated
num_experts = w1.shape[0]
dtype = hidden_states.dtype
device = hidden_states.device
topk_weights = topk_weights.to(dtype)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this fix is done by @whx-sjtu as a part in #863, but I think it should be merge quickly instead of blocking in #863

@MengqingCao MengqingCao force-pushed the dsfix branch 2 times, most recently from cb2ee7d to 0067a94 Compare May 22, 2025 11:33
@MengqingCao MengqingCao mentioned this pull request May 26, 2025
Closed
@MengqingCao MengqingCao force-pushed the dsfix branch 2 times, most recently from a57bdd4 to 2c38964 Compare May 28, 2025 01:16
wangxiyuan
wangxiyuan reviewed May 28, 2025
View reviewed changes
@MengqingCao MengqingCao force-pushed the dsfix branch 3 times, most recently from 9f976dd to d1a117a Compare May 30, 2025 03:13
@MengqingCao MengqingCao added long-term-test enable long term test for PR ready-for-test start test by label for PR labels May 30, 2025
@MengqingCao MengqingCao added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR labels Jun 4, 2025
@wangxiyuan wangxiyuan mentioned this pull request Jun 4, 2025
Closed
76 tasks
@MengqingCao MengqingCao changed the title [Bugfix] Fix deepseek V0 percision issue and add acc ci for it [Bugfix] Fix deepseek percision issue and add acc ci for it Jun 4, 2025
@MengqingCao MengqingCao force-pushed the dsfix branch 2 times, most recently from ba23af4 to 988ab44 Compare June 4, 2025 08:42
@github-actions
Copy link

github-actions bot commented Jun 4, 2025

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it
5c86b54 
Signed-off-by: MengqingCao <cmq0113@163.com>
@Yikun Yikun added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR labels Jun 4, 2025
@MengqingCao MengqingCao added ready-for-test start test by label for PR and removed ready-for-test start test by label for PR labels Jun 4, 2025
Yikun
Yikun reviewed Jun 4, 2025
View reviewed changes
Copy link
Collaborator

@Yikun Yikun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge this first, if e2e CI passed to make main CI recover.

And address comments and recover long term CI in sperate PR.

.github/workflows/vllm_ascend_test_long_term.yaml
matrix:
os: [linux-arm64-npu-1, linux-arm64-npu-4]
vllm_version: [main, v0.9.0]
concurrency:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems not work, pls try to change global concurrency

.github/workflows/vllm_ascend_test_long_term.yaml
pytest -sv tests/long_term/spec_decode --ignore=tests/long_term/spec_decode/e2e/test_mtp_correctness.py --ignore=tests/long_term/spec_decode/e2e/test_v1_spec_decode.py --ignore=tests/long_term/spec_decode/e2e/test_v1_mtp_correctness.py
pytest -sv tests/long_term/test_accuracy.py
else
VLLM_USE_MODELSCOPE=True pytest -sv tests/long_term/test_deepseek_v2_lite_tp2_accuracy.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could move to multicard because gsm8k it's ok to run every PR.

tests/long_term/test_deepseek_v2_lite_tp2_accuracy.py
RTOL = 0.03
# Baseline accuracy after VLLM optimization.
# FIXME: fix the accuracy issue
EXPECTED_VALUE = 0.000758150113722517
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shoudl be fixed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, will do it in next pr

@Yikun Yikun merged commit afc4c0c into vllm-project:main Jun 4, 2025
26 of 30 checks passed
@Yikun
Copy link
Collaborator

Yikun commented Jun 4, 2025

Merged to main to recover main CI

@MengqingCao MengqingCao deleted the dsfix branch June 4, 2025 12:33
wanghanqingLYT pushed a commit to raindaywhu/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao @wanghanqingLYT
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
437bf63 
…ject#905)

Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
weijinqian0 pushed a commit to weijinqian0/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
9d31166 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
weijinqian0 pushed a commit to weijinqian0/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
46813cc 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
weijinqian0 pushed a commit to weijinqian0/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
bbbc3f8 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
weijinqian0 pushed a commit to weijinqian0/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
d9f3257 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
weijinqian0 pushed a commit to weijinqian0/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
9ffebf7 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
faf5ee4 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
5b2b1c9 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
db01521 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
39bef09 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
11b1a3b 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 4, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
0d86246 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 5, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
5a1b3e3 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 5, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
91e7c64 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 5, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
5f63e62 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 5, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
c2f0ab8 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 5, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
df7bd60 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: wangxiaoxin (A) <w00664509@china.huawei.com>
@MengqingCao MengqingCao mentioned this pull request Jun 12, 2025
Open
83 tasks
venus-taibai pushed a commit to venus-taibai/vllm-ascend that referenced this pull request Jun 18, 2025
@ApsarasX
PullRequest: 107 [Bugfix] Fix deepseek percision issue and add acc ci…
1f413aa 
… for it (vllm-project#905)

Merge branch wengang/cherry-pick-905 of git@code.alipay.com:Theta/vllm-ascend.git into dev-v0.9.0604
https://code.alipay.com/Theta/vllm-ascend/pull_requests/107

Reviewed-by: 子宏 <tanzhiqiang.tzq@antgroup.com>


* [Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-project#905)
chopper0126 pushed a commit to chopper0126/vllm-ascend that referenced this pull request Oct 16, 2025
@MengqingCao
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
c748c3d 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Angazenn pushed a commit to Angazenn/vllm-ascend that referenced this pull request Oct 21, 2025
@MengqingCao @Angazenn
[Bugfix] Fix deepseek percision issue and add acc ci for it (vllm-pro…
455facd 
…ject#905)

### What this PR does / why we need it?
Fix deepseek percision issue on V0 and add acc ci for it
Fixes vllm-project#1062
### How was this patch tested?
CI passed with new added test.

Signed-off-by: MengqingCao <cmq0113@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wangxiyuan wangxiyuan wangxiyuan left review comments

@ganyi1996ppo ganyi1996ppo ganyi1996ppo left review comments

@Yikun Yikun Yikun approved these changes

Assignees

No one assigned

Labels

long-term-test enable long term test for PR module:ops module:quantization module:tests ready-for-test start test by label for PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Bug]: deepseek-v2-lite offline failed to run

4 participants

@MengqingCao @Yikun @wangxiyuan @ganyi1996ppo