[CI/UT][Refactor] move e2e spec decode and deepseek acc test to per pr #1136

MengqingCao · 2025-06-09T09:43:21Z

What this PR does / why we need it?

run deepseek acc ut per pr --- multicard CI time increased by 9 min
run spec decode e2e test on v1 per pr --- singlecard CI time increased by 3 min (partly is disabled due to not work now)
~~3. align the output of whether dbo is enabled or not~~
The generated results with and without dbo cannot be aligned.
https://github.com/vllm-project/vllm-ascend/actions/runs/15822900528/job/44600029405?pr=1136
skip V0 mtp test due to failure in https://github.com/vllm-project/vllm-ascend/actions/runs/16012172833/job/45171988816
fix some version conflicts

How was this patch tested?

CI passed with new added test.

MengqingCao · 2025-06-09T09:45:09Z

tests/multicard/test_offline_inference_distributed.py

        vllm_model.generate_greedy(example_prompts, max_tokens)


-def test_models_distributed_DeepSeek():


I think there is no need to run e2e functional ut as we already have acc ut

no, acc test belongs to long-term-test, it is not ran in every commit.

This won't take a long time as dataset gsmk is small. wydt? cc @Yikun

github-actions · 2025-06-11T01:20:57Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2025-06-11T01:20:57Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

MengqingCao · 2025-06-20T06:41:44Z

.github/workflows/vllm_ascend_test.yaml

+          --ignore=tests/e2e/singlecard/long_term/spec_decode/e2e/test_v1_mtp_correctness.py
+          # ------------ spec decode e2e test on v1 ------------ #
+          VLLM_USE_MODELSCOPE=True pytest -sv tests/e2e/singlecard/long_term/spec_decode/e2e/test_v1_mtp_correctness.py
+          # TODO: revert me when test_v1_spec_decode.py::test_ngram_correctness is fixed


test_v1_spec_decode.py::test_ngram_correctness is fixed in #1189. Will revert this when #1189 merged

github-actions · 2025-06-20T09:22:48Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

codecov · 2025-06-23T03:18:33Z

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 52.36%. Comparing base (c30ddb8) to head (211eabd).
⚠️ Report is 613 commits behind head on main.

Files with missing lines	Patch %	Lines
vllm_ascend/ops/fused_moe.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1136       +/-   ##
===========================================
+ Coverage   27.39%   52.36%   +24.96%     
===========================================
  Files          56       78       +22     
  Lines        6191     9631     +3440     
===========================================
+ Hits         1696     5043     +3347     
- Misses       4495     4588       +93

Flag	Coverage Δ
unittests	`52.36% <66.66%> (+24.96%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

MengqingCao · 2025-06-23T13:25:03Z

The generated results with and without dbo cannot be aligned.
https://github.com/vllm-project/vllm-ascend/actions/runs/15822900528/job/44600029405?pr=1136

MengqingCao · 2025-06-25T03:01:20Z

This pr is ready for review cc @Yikun @wangxiyuan

github-actions · 2025-06-30T08:36:47Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

### What this PR does / why we need it? mla attention still using the gpu_input_batch's attr:`swap_states`, which will lead to an error `AttributeError: 'InputBatch' object has no attribute 'swap_states'` This PR fixed the mla input patch error ### How was this patch tested? will be tested by #1136 --------- Signed-off-by: wangli <wangli858794774@gmail.com>

github-actions · 2025-07-02T09:48:25Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

### What this PR does / why we need it? mla attention still using the gpu_input_batch's attr:`swap_states`, which will lead to an error `AttributeError: 'InputBatch' object has no attribute 'swap_states'` This PR fixed the mla input patch error ### How was this patch tested? will be tested by vllm-project#1136 --------- Signed-off-by: wangli <wangli858794774@gmail.com> Signed-off-by: ZhengWG <zwg0606@gmail.com>

* move e2e spec decode and deepseek acc test to per pr * move test_fused_moe_allgather_ep.py to e2e/multicard * remove e2e test on deepseek-v2-lite due to already test acc Signed-off-by: MengqingCao <cmq0113@163.com>

Signed-off-by: MengqingCao <cmq0113@163.com>

### What this PR does / why we need it? mla attention still using the gpu_input_batch's attr:`swap_states`, which will lead to an error `AttributeError: 'InputBatch' object has no attribute 'swap_states'` This PR fixed the mla input patch error ### How was this patch tested? will be tested by vllm-project#1136 --------- Signed-off-by: wangli <wangli858794774@gmail.com>

vllm-project#1136) ### What this PR does / why we need it? 1. run deepseek acc ut per pr --- multicard CI time increased by 9 min 2. run spec decode e2e test on v1 per pr --- singlecard CI time increased by 3 min (partly is disabled due to not work now) ~~3. align the output of whether dbo is enabled or not~~ The generated results with and without dbo cannot be aligned. https://github.com/vllm-project/vllm-ascend/actions/runs/15822900528/job/44600029405?pr=1136 4. skip V0 mtp test due to failure in https://github.com/vllm-project/vllm-ascend/actions/runs/16012172833/job/45171988816 5. fix some version conflicts ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com>

### What this PR does / why we need it? mla attention still using the gpu_input_batch's attr:`swap_states`, which will lead to an error `AttributeError: 'InputBatch' object has no attribute 'swap_states'` This PR fixed the mla input patch error ### How was this patch tested? will be tested by vllm-project#1136 --------- Signed-off-by: wangli <wangli858794774@gmail.com>

vllm-project#1136) ### What this PR does / why we need it? 1. run deepseek acc ut per pr --- multicard CI time increased by 9 min 2. run spec decode e2e test on v1 per pr --- singlecard CI time increased by 3 min (partly is disabled due to not work now) ~~3. align the output of whether dbo is enabled or not~~ The generated results with and without dbo cannot be aligned. https://github.com/vllm-project/vllm-ascend/actions/runs/15822900528/job/44600029405?pr=1136 4. skip V0 mtp test due to failure in https://github.com/vllm-project/vllm-ascend/actions/runs/16012172833/job/45171988816 5. fix some version conflicts ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao commented Jun 9, 2025

View reviewed changes

github-actions bot added module:tests merge-conflicts labels Jun 9, 2025

MengqingCao force-pushed the dsci branch from 927c586 to 1469d79 Compare June 19, 2025 06:58

github-actions bot removed the merge-conflicts label Jun 19, 2025

MengqingCao changed the title ~~[CI] Run deepseek acc ut per pr~~ [CI/UT][Refactor] move e2e spec decode and deepseek acc test to per pr Jun 19, 2025

MengqingCao force-pushed the dsci branch from 1fa2ce4 to 30c7898 Compare June 20, 2025 01:26

MengqingCao commented Jun 20, 2025

View reviewed changes

github-actions bot added the merge-conflicts label Jun 20, 2025

MengqingCao force-pushed the dsci branch from 0b5f317 to 0b07aa0 Compare June 23, 2025 02:28

github-actions bot removed the merge-conflicts label Jun 23, 2025

MengqingCao force-pushed the dsci branch from 2882fe7 to c7f3b7d Compare June 23, 2025 03:03

MengqingCao force-pushed the dsci branch from ad1fb75 to c0134d2 Compare June 23, 2025 06:22

MengqingCao force-pushed the dsci branch from 52bbbf1 to f780678 Compare June 23, 2025 13:36

MengqingCao force-pushed the dsci branch 2 times, most recently from eb46671 to 20bb139 Compare June 28, 2025 10:36

github-actions bot added the merge-conflicts label Jun 30, 2025

MengqingCao force-pushed the dsci branch from 20bb139 to b2d63bf Compare June 30, 2025 11:21

github-actions bot removed the merge-conflicts label Jun 30, 2025

MengqingCao force-pushed the dsci branch 2 times, most recently from d659d4c to 410650c Compare July 2, 2025 04:56

MengqingCao mentioned this pull request Jul 2, 2025

[Bugfix] Add func swap_states to fix MLA attention #1580

Merged

github-actions bot added the merge-conflicts label Jul 2, 2025

MengqingCao force-pushed the dsci branch from 410650c to 8f47a03 Compare July 2, 2025 10:12

github-actions bot removed the merge-conflicts label Jul 2, 2025

MengqingCao added 2 commits July 4, 2025 02:43

[CI/UT][Refactor] Some refactors on UT

fabc767

* move e2e spec decode and deepseek acc test to per pr * move test_fused_moe_allgather_ep.py to e2e/multicard * remove e2e test on deepseek-v2-lite due to already test acc Signed-off-by: MengqingCao <cmq0113@163.com>

skip V0 mtp test

59d68fe

Signed-off-by: MengqingCao <cmq0113@163.com>

MengqingCao force-pushed the dsci branch from 27029e5 to 59d68fe Compare July 4, 2025 02:43

MengqingCao added 2 commits July 4, 2025 04:32

fix spec-decode

726864f

Signed-off-by: MengqingCao <cmq0113@163.com>

fix FusedMoEParallelConfig

211eabd

Signed-off-by: MengqingCao <cmq0113@163.com>

github-actions bot added the module:ops label Jul 4, 2025

Yikun approved these changes Jul 4, 2025

View reviewed changes

wangxiyuan approved these changes Jul 4, 2025

View reviewed changes

wangxiyuan merged commit dd22ac3 into vllm-project:main Jul 4, 2025
20 checks passed

MengqingCao deleted the dsci branch July 8, 2025 02:14

Yikun mentioned this pull request Jul 13, 2025

[Bug]: test_ngram_correctness failed due to PagedAttentionOperation inner error #1162

Open

		vllm_model.generate_greedy(example_prompts, max_tokens)


		def test_models_distributed_DeepSeek():

[CI/UT][Refactor] move e2e spec decode and deepseek acc test to per pr #1136

[CI/UT][Refactor] move e2e spec decode and deepseek acc test to per pr #1136

Uh oh!

Conversation

MengqingCao commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

How was this patch tested?

Uh oh!

MengqingCao Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

wangxiyuan Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

MengqingCao Jun 9, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 11, 2025

Uh oh!

github-actions bot commented Jun 11, 2025

Uh oh!

MengqingCao Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 20, 2025

Uh oh!

codecov bot commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

MengqingCao commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MengqingCao commented Jun 25, 2025

Uh oh!

github-actions bot commented Jun 30, 2025

Uh oh!

github-actions bot commented Jul 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MengqingCao commented Jun 9, 2025 •

edited

Loading

codecov bot commented Jun 23, 2025 •

edited

Loading

MengqingCao commented Jun 23, 2025 •

edited

Loading