Skip to content

[Feature] Enable inference support for Deepseekr1-w8a8-MTP #1994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 29, 2025
Merged

[Feature] Enable inference support for Deepseekr1-w8a8-MTP #1994

merged 1 commit into from
Jul 29, 2025

Conversation

@Irving11-BKN
Copy link
Contributor

@Irving11-BKN Irving11-BKN commented Jul 24, 2025 •
edited by github-actions bot
Loading

Support the inference of the Deepseekr1-w8a8-mtp model with statically-quantized shared_head in MTP layers.

Signed-off-by: curryliu [120010041@link.cuhk.edu.cn]

@Irving11-BKN
�[200~[Feature] Enable inference support for Deepseekr1-w8a8-MTP
ad4fd7c 
Signed-off-by: curryliu <120010041@link.cuhk.edu.cn>
@codecov
Copy link

codecov bot commented Jul 24, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 37.50000% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 71.43%. Comparing base (ff97740) to head (ad4fd7c).
⚠️ Report is 638 commits behind head on main.

Files with missing lines Patch % Lines
vllm_ascend/quantization/quant_config.py 33.33% 6 Missing ⚠️
vllm_ascend/models/deepseek_mtp.py 42.85% 4 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1994      +/-   ##
==========================================
- Coverage   71.49%   71.43%   -0.06%     
==========================================
  Files          86       86              
  Lines        9131     9145      +14     
==========================================
+ Hits         6528     6533       +5     
- Misses       2603     2612       +9
Flag Coverage Δ
unittests 71.43% <37.50%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jianzs jianzs merged commit ca8007f into vllm-project:main Jul 29, 2025
24 checks passed
weijinqian0 pushed a commit to weijinqian0/vllm-ascend that referenced this pull request Jul 30, 2025
@Irving11-BKN
[Feature] Enable inference support for Deepseekr1-w8a8-MTP (vllm-proj…
70c011e 
…ect#1994)

Support the inference of the Deepseekr1-w8a8-mtp model with
statically-quantized shared_head in MTP layers.

- vLLM version: v0.9.2
- vLLM main:
vllm-project/vllm@6eca337

Signed-off-by: curryliu <120010041@link.cuhk.edu.cn>
weijinqian0 pushed a commit to weijinqian0/vllm-ascend that referenced this pull request Jul 30, 2025
@Irving11-BKN
[Feature] Enable inference support for Deepseekr1-w8a8-MTP (vllm-proj…
05f2ff2 
…ect#1994)

Support the inference of the Deepseekr1-w8a8-mtp model with
statically-quantized shared_head in MTP layers.

- vLLM version: v0.9.2
- vLLM main:
vllm-project/vllm@6eca337

Signed-off-by: curryliu <120010041@link.cuhk.edu.cn>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
This was referenced Aug 1, 2025
Closed
Merged
wangxiyuan pushed a commit that referenced this pull request Aug 6, 2025
@zhoux77899
[main][Bugfix] Fix unable to load qwen3_moe quantized weights (#2219)
e31b31f 
### What this PR does / why we need it?

Fixes unable to load `qwen3_moe` quantized weights issue due to #1994

### Does this PR introduce _any_ user-facing change?

None

### How was this patch tested?

Add a `qwen3_moe` W8A8 quantized model in
`tests/e2e/multicard/test_qwen3_moe.py`

- vLLM version: v0.10.0
- vLLM main:
vllm-project/vllm@c494f96

---------

Signed-off-by: zhoux77899 <zhouxiang100@huawei.com>
zzhx1 pushed a commit to lidenghui1110/vllm-ascend that referenced this pull request Aug 11, 2025
@zhoux77899
[main][Bugfix] Fix unable to load qwen3_moe quantized weights (vllm-p…
2f01e2e 
…roject#2219)

### What this PR does / why we need it?

Fixes unable to load `qwen3_moe` quantized weights issue due to vllm-project#1994

### Does this PR introduce _any_ user-facing change?

None

### How was this patch tested?

Add a `qwen3_moe` W8A8 quantized model in
`tests/e2e/multicard/test_qwen3_moe.py`

- vLLM version: v0.10.0
- vLLM main:
vllm-project/vllm@c494f96

---------

Signed-off-by: zhoux77899 <zhouxiang100@huawei.com>
zzhx1 pushed a commit to lidenghui1110/vllm-ascend that referenced this pull request Aug 11, 2025
@zhoux77899 @zzhx1
[main][Bugfix] Fix unable to load qwen3_moe quantized weights (vllm-p…
e03f066 
…roject#2219)

### What this PR does / why we need it?

Fixes unable to load `qwen3_moe` quantized weights issue due to vllm-project#1994

### Does this PR introduce _any_ user-facing change?

None

### How was this patch tested?

Add a `qwen3_moe` W8A8 quantized model in
`tests/e2e/multicard/test_qwen3_moe.py`

- vLLM version: v0.10.0
- vLLM main:
vllm-project/vllm@c494f96

---------

Signed-off-by: zhoux77899 <zhouxiang100@huawei.com>
chopper0126 pushed a commit to chopper0126/vllm-ascend that referenced this pull request Sep 26, 2025
@Irving11-BKN
[Feature] Enable inference support for Deepseekr1-w8a8-MTP (vllm-proj…
1546e09 
…ect#1994)

Support the inference of the Deepseekr1-w8a8-mtp model with
statically-quantized shared_head in MTP layers.

- vLLM version: v0.9.2
- vLLM main:
vllm-project/vllm@6eca337

Signed-off-by: curryliu <120010041@link.cuhk.edu.cn>
chopper0126 pushed a commit to chopper0126/vllm-ascend that referenced this pull request Sep 26, 2025
@zhoux77899
[main][Bugfix] Fix unable to load qwen3_moe quantized weights (vllm-p…
848e7c8 
…roject#2219)

### What this PR does / why we need it?

Fixes unable to load `qwen3_moe` quantized weights issue due to vllm-project#1994

### Does this PR introduce _any_ user-facing change?

None

### How was this patch tested?

Add a `qwen3_moe` W8A8 quantized model in
`tests/e2e/multicard/test_qwen3_moe.py`

- vLLM version: v0.10.0
- vLLM main:
vllm-project/vllm@c494f96

---------

Signed-off-by: zhoux77899 <zhouxiang100@huawei.com>
Angazenn pushed a commit to Angazenn/vllm-ascend that referenced this pull request Oct 21, 2025
@Irving11-BKN @Angazenn
[Feature] Enable inference support for Deepseekr1-w8a8-MTP (vllm-proj…
b8250d8 
…ect#1994)

Support the inference of the Deepseekr1-w8a8-mtp model with
statically-quantized shared_head in MTP layers.

- vLLM version: v0.9.2
- vLLM main:
vllm-project/vllm@6eca337

Signed-off-by: curryliu <120010041@link.cuhk.edu.cn>
Angazenn pushed a commit to Angazenn/vllm-ascend that referenced this pull request Oct 21, 2025
@zhoux77899 @Angazenn
[main][Bugfix] Fix unable to load qwen3_moe quantized weights (vllm-p…
e698167 
…roject#2219)

### What this PR does / why we need it?

Fixes unable to load `qwen3_moe` quantized weights issue due to vllm-project#1994

### Does this PR introduce _any_ user-facing change?

None

### How was this patch tested?

Add a `qwen3_moe` W8A8 quantized model in
`tests/e2e/multicard/test_qwen3_moe.py`

- vLLM version: v0.10.0
- vLLM main:
vllm-project/vllm@c494f96

---------

Signed-off-by: zhoux77899 <zhouxiang100@huawei.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wangxiyuan wangxiyuan wangxiyuan approved these changes

Assignees

No one assigned

Labels

module:quantization

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Irving11-BKN @wangxiyuan @jianzs