[Bug]: When moe ep=16 etp=1, the result is normal. When moe ep=1 etp=16, the result is abnormal.

[ttanzhiqiang]

Your current environment

nohup python -m vllm.entrypoints.openai.api_server --model=/mnt/deepseek/DeepSeek-R1-W8A8-VLLM \ --trust-remote-code \ --distributed-executor-backend=mp \ -tp=16 \ -dp=1 \ --port 8006 \ --max-num-seqs 24 \ --max-model-len 32768 \ --max-num-batched-tokens 32768 \ --block-size 128 \ --enable-expert-parallel \ --compilation_config 0 \ --gpu-memory-utilization 0.96 \ --additional-config '{"expert_tensor_parallel_size":1, "ascend_scheduler_config":{}}' &> run.log &

nohup python -m vllm.entrypoints.openai.api_server --model=/mnt/deepseek/DeepSeek-R1-W8A8-VLLM
--trust-remote-code
--distributed-executor-backend=mp
-tp=16
-dp=1
--port 8006
--max-num-seqs 24
--max-model-len 32768
--max-num-batched-tokens 32768
--block-size 128
--enable-expert-parallel
--compilation_config 0
--gpu-memory-utilization 0.96
--additional-config '{"expert_tensor_parallel_size":16, "ascend_scheduler_config":{}}' &> run.log &

The principle of the problem：In the case of etp16 there is no parallel processing

🐛 Describe the bug

nohup python -m vllm.entrypoints.openai.api_server --model=/mnt/deepseek/DeepSeek-R1-W8A8-VLLM
--trust-remote-code
--distributed-executor-backend=mp
-tp=16
-dp=1
--port 8006
--max-num-seqs 24
--max-model-len 32768
--max-num-batched-tokens 32768
--block-size 128
--enable-expert-parallel
--compilation_config 0
--gpu-memory-utilization 0.96
--additional-config '{"expert_tensor_parallel_size":16, "ascend_scheduler_config":{}}' &> run.log &

[ttanzhiqiang]

@wangxiyuan @Yikun

[ttanzhiqiang]

vllm_ascend commit id:01e3d59eae28be39df772dfb6c0113bf9e1552e9
vllm: v0.8.5

[ttanzhiqiang]

@Angazenn

[Angazenn]

This problem might be related to this PR #863. You can have a try. @ttanzhiqiang

[ttanzhiqiang]

ok

[ttanzhiqiang]

This problem might be related to this PR #863. You can have a try. @ttanzhiqiang
@Angazenn I tried this PR, which should fix the bug of etp16 before all2all, and I can use it. But now it has been changed to the all2all method, so this PR should not be used anymore

[ttanzhiqiang]

#985 This PR will fix this problem

[Yikun]

Thanks all