Skip to content

Conversation

@mengwei805
Copy link
Collaborator

@mengwei805 mengwei805 commented Jun 20, 2025

What this PR does / why we need it?

  1. PR913 introduced an error that caused V0's spec decode function to fail. PR1109 wanted to fix this problem. Unfortunately, the fix broke the ngram function. I fixed the ngram function in this PR. PS: Q: Why is there a problem when ngram is not found when pr1109 is merged? A: The newly introduced problem will only appear when tp>1, and the use cases on CI are all tp=1
  2. In versions after 0.7.3, vllm-ascend deleted some spec decode UTs to avoid CI taking too long, including eagle speculative UTs, which made CI unable to take care of the eagle function. I added it(test_eagle_correctness.py) back in this PR
  3. Because of the reason mentioned in 2, the current version of Eagle has a problem. I located and fixed this problem. It was because vllm's draft_model_runner.py was changed and vllm-ascend was not synchronized in time.
  4. Currently, the UTs of v0 and v1 are mixed in the spec_decode directory. I split them into two directories: spec_decode_v0 and spec_decode_v1.
  5. i found vllm.spec_decode.multi_step_worker.MultiStepWorker.set_include_gpu_probs_tensor and vllm.spec_decode.multi_step_worker.MultiStepWorker.set_should_modify_greedy_probs_inplace have changed in vllm, so i remove its patchs in this pr.
  6. v1 mtp ut failed(https://github.com/vllm-project/vllm-ascend/actions/runs/15782006176/job/44489813330?pr=1323),
    I commented it out. @XWFAlone @JC-ut0

Does this PR introduce any user-facing change?

This PR fixes the functions of ngram and eagle spec decode in the v0 engine

How was this patch tested?

ngram and eagle were tested locally using an 800I A2 machine, using real weights instead of the random small weights used by UT, and using a scenario test with tp>1.
and other were tested by CI

@mengwei805 mengwei805 force-pushed the v0_spec_bugfix_091 branch 2 times, most recently from cfd4207 to 71e5adc Compare June 20, 2025 13:29
@mengwei805 mengwei805 added ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR and removed ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR labels Jun 20, 2025
@mengwei805 mengwei805 force-pushed the v0_spec_bugfix_091 branch from 71e5adc to 30f24a2 Compare June 20, 2025 15:07
@mengwei805 mengwei805 added ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR and removed ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR labels Jun 20, 2025
@mengwei805 mengwei805 force-pushed the v0_spec_bugfix_091 branch from 30f24a2 to 98ab1c7 Compare June 20, 2025 16:55
@mengwei805 mengwei805 added ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR and removed ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR labels Jun 20, 2025
@mengwei805 mengwei805 force-pushed the v0_spec_bugfix_091 branch from 98ab1c7 to e3a4b19 Compare June 20, 2025 17:11
@mengwei805 mengwei805 removed ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR labels Jun 20, 2025
@mengwei805 mengwei805 added ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR and removed ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR labels Jun 20, 2025
Signed-off-by: mengwei805 <mengwei25@huawei.com>
@mengwei805 mengwei805 force-pushed the v0_spec_bugfix_091 branch from 3d98732 to c1346cc Compare June 21, 2025 02:04
@mengwei805 mengwei805 added ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR and removed ready read for review long-term-test enable long term test for PR ready-for-test start test by label for PR labels Jun 21, 2025
@ganyi1996ppo ganyi1996ppo merged commit 85aa6c8 into vllm-project:v0.9.1-dev Jun 21, 2025
24 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

long-term-test enable long term test for PR module:tests ready read for review ready-for-test start test by label for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants