Skip to content

Commit 2008152

Browse files
zhanghw0354zhanghaiwen
andauthored
[main][bugfix]Fix vLLM startup failure when inferring DeepSeek R1 model in DP scenario (#2020)
### What this PR does / why we need it? Fix vLLM startup failure when inferring DeepSeek R1 model in DP scenario. When running vLLM inference for the DeepSeek R1 model in DP32+TP1 configuration, the vLLM service fails to start with the following error. <img width="1786" height="918" alt="21b2011042d4f77f36f5243fa64d9c18" src="https://github.com/user-attachments/assets/df1963fe-587e-43ca-822e-a9094d0034fb" /> The root cause is a missing else branch after [this line of code](https://github.com/vllm-project/vllm-ascend/blob/d629f0b2b573c3ba858a09fc93c42f2c2634e043/vllm_ascend/ops/fused_moe.py#L1411). This PR fixes the issue. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? CI passed with new added/existing test. - vLLM version: v0.10.0 - vLLM main: vllm-project/vllm@5bbaf49 --------- Signed-off-by: zhanghaiwen <zhanghaiwen@cmss.chinamobile.com> Co-authored-by: zhanghaiwen <zhanghaiwen@cmss.chinamobile.com>
1 parent 7c90ba5 commit 2008152

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

vllm_ascend/ops/fused_moe.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1469,6 +1469,8 @@ def forward(self,
14691469
e_hidden_states, dim=0)
14701470
final_hidden_states = final_hidden_states[:num_tokens]
14711471
dispose_tensor(e_hidden_states)
1472+
else:
1473+
final_hidden_states = e_hidden_states
14721474
else:
14731475
final_hidden_states = e_hidden_states
14741476

0 commit comments

Comments
 (0)