[v0.8.5rc1] FAQ / Feedback | 问题/反馈

[Yikun]

Please use doc: https://vllm-ascend.readthedocs.io

---

请使用 https://vllm-ascend.readthedocs.io 安装

[BestKuan]

Qwen3-235B-A22B can't run successful when setting VLLM_USE_V1=1 and v0 engine is ok. #781

[gao12312]

can not run deepseek-R1 w8a8 , #813

RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.43.down_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.gate_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.gate_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.up_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.43.up_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.43.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.44.down_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.44.down_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.down_proj.,param_name is experts.w2_,name is model.layers.58.mlp.experts.w2_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.gate_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.gate_proj.weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.gate_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight_scale
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.up_proj.weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:795] after weight_name is experts.44.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.w13_weight
(RayWorkerWrapper pid=332882) INFO 05-13 10:04:15 [deepseek_v2.py:791] before weight_name is experts.44.up_proj.,param_name is experts.w13_,name is model.layers.58.mlp.experts.44.up_proj.weight_scale
Loading safetensors checkpoint shards:   1% Completed | 1/157 [00:00<00:43,  3.56it/s]
Loading safetensors checkpoint shards:   1% Completed | 2/157 [00:00<00:43,  3.55it/s]
ERROR 05-13 10:04:16 [core.py:396] EngineCore failed to start.
ERROR 05-13 10:04:16 [core.py:396] Traceback (most recent call last):
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 387, in run_engine_core
ERROR 05-13 10:04:16 [core.py:396]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 329, in __init__
ERROR 05-13 10:04:16 [core.py:396]     super().__init__(vllm_config, executor_class, log_stats,
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 64, in __init__
ERROR 05-13 10:04:16 [core.py:396]     self.model_executor = executor_class(vllm_config)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 286, in __init__
ERROR 05-13 10:04:16 [core.py:396]     super().__init__(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 52, in __init__
ERROR 05-13 10:04:16 [core.py:396]     self._init_executor()
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/ray_distributed_executor.py", line 114, in _init_executor
ERROR 05-13 10:04:16 [core.py:396]     self._init_workers_ray(placement_group)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/ray_distributed_executor.py", line 396, in _init_workers_ray
ERROR 05-13 10:04:16 [core.py:396]     self._run_workers("load_model",
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/ray_distributed_executor.py", line 521, in _run_workers
ERROR 05-13 10:04:16 [core.py:396]     ray_worker_outputs = ray.get(ray_worker_outputs)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
ERROR 05-13 10:04:16 [core.py:396]     return fn(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
ERROR 05-13 10:04:16 [core.py:396]     return func(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/worker.py", line 2822, in get
ERROR 05-13 10:04:16 [core.py:396]     values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
ERROR 05-13 10:04:16 [core.py:396]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/ray/_private/worker.py", line 930, in get_objects
ERROR 05-13 10:04:16 [core.py:396]     raise value.as_instanceof_cause()
ERROR 05-13 10:04:16 [core.py:396] ray.exceptions.RayTaskError(KeyError): ray::RayWorkerWrapper.execute_method() (pid=64195, ip=10.151.18.104, actor_id=e36a0df3031e94cd0ece267115000000, repr=<vllm.executor.ray_utils.RayWorkerWrapper object at 0xfffc23b6fd00>)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/worker/worker_base.py", line 621, in execute_method
ERROR 05-13 10:04:16 [core.py:396]     raise e
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/worker/worker_base.py", line 612, in execute_method
ERROR 05-13 10:04:16 [core.py:396]     return run_method(self, method, args, kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/utils.py", line 2456, in run_method
ERROR 05-13 10:04:16 [core.py:396]     return func(*args, **kwargs)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 178, in load_model
ERROR 05-13 10:04:16 [core.py:396]     self.model_runner.load_model()
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 939, in load_model
ERROR 05-13 10:04:16 [core.py:396]     self.model = get_model(vllm_config=self.vllm_config)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/__init__.py", line 14, in get_model
ERROR 05-13 10:04:16 [core.py:396]     return loader.load_model(vllm_config=vllm_config)
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/loader.py", line 455, in load_model
ERROR 05-13 10:04:16 [core.py:396]     loaded_weights = model.load_weights(
ERROR 05-13 10:04:16 [core.py:396]   File "/vllm-workspace/vllm/vllm/model_executor/models/deepseek_v2.py", line 788, in load_weights
ERROR 05-13 10:04:16 [core.py:396]     param = params_dict[name]
ERROR 05-13 10:04:16 [core.py:396] KeyError: 'model.layers.57.mlp.experts.w2_weight_offset'

[wphtrying]

问下安装v0.8.5rc1 版本的vllm-ascend，一定需要gcc高于9吗？

[Yikun]

@wphtrying according our real test, it should >= gcc8.

[marxy]

I run Qwen3-30B-A3B on 910b2. It's very slow with only 7 tokens per second. How can I optimize it to improve performance.

[1103036128]

NFO 05-16 07:03:36 [async_llm_engine.py:211] Added request chatcmpl-8bcb892f00d64ac78eadc200c363eca5.
INFO 05-16 07:03:36 [metrics.py:486] Avg prompt throughput: 1.7 tokens/s, Avg generation throughput: 0.2 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 05-16 07:03:41 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 05-16 07:03:47 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 05-16 07:03:52 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 05-16 07:03:57 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4.4 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 05-16 07:04:02 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4.1 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 05-16 07:04:07 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 4.3 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.1%, CPU KV cache usage: 0.0%.
INFO 05-16 07:04:09 [async_llm_engine.py:179] Finished request chatcmpl-8bcb892f00d64ac78eadc200c363eca5.
INFO 05-16 07:04:22 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.5 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 05-16 07:04:32 [metrics.py:486] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.

[1103036128]

Qwen3-235B reasoning speed is too slow. What good way is there?

[zqWu]

fail to start 0.8.5rc1 + qwen3-32b. this is the error log,

(VllmWorker rank=0 pid=565) INFO 05-19 02:31:35 [parallel_state.py:1004] rank 0 in world size 2 is assigned as DP rank 0, PP rank 0, TP rank 0
(VllmWorker rank=1 pid=566) INFO 05-19 02:31:35 [parallel_state.py:1004] rank 1 in world size 2 is assigned as DP rank 0, PP rank 0, TP rank 1
(VllmWorker rank=1 pid=566) INFO 05-19 02:31:36 [model_runner_v1.py:852] Starting to load model /data1/models/Qwen3-32B...
(VllmWorker rank=0 pid=565) INFO 05-19 02:31:36 [model_runner_v1.py:852] Starting to load model /data1/models/Qwen3-32B...
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435] WorkerProc failed to start.
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435] Traceback (most recent call last):
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 409, in worker_main
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     worker = WorkerProc(*args, **kwargs)
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 306, in __init__
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     self.worker.load_model()
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 178, in load_model
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     self.model_runner.load_model()
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 855, in load_model
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     self.model = get_model(vllm_config=self.vllm_config)
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/__init__.py", line 14, in get_model
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     return loader.load_model(vllm_config=vllm_config)
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/loader.py", line 452, in load_model
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     model = _initialize_model(vllm_config=vllm_config)
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/model_executor/model_loader/loader.py", line 133, in _initialize_model
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     return model_class(vllm_config=vllm_config, prefix=prefix)
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen3.py", line 269, in __init__
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     self.model = Qwen3Model(vllm_config=vllm_config,
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/compilation/decorators.py", line 151, in __init__
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs)
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen3.py", line 241, in __init__
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     super().__init__(vllm_config=vllm_config,
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/compilation/decorators.py", line 162, in __init__
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     TorchCompileWrapperWithCustomDispatcher.__init__(
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/compilation/wrapper.py", line 42, in __init__
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     backend = vllm_config.compilation_config.init_backend(vllm_config)
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/config.py", line 3600, in init_backend
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from vllm.compilation.backends import VllmBackend
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/compilation/backends.py", line 20, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from .compiler_interface import EagerAdaptor, InductorAdaptor
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/vllm-workspace/vllm/vllm/compilation/compiler_interface.py", line 11, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     import torch._inductor.compile_fx
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/compile_fx.py", line 72, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from .fx_passes.joint_graph import joint_graph_passes
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/fx_passes/joint_graph.py", line 19, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from ..pattern_matcher import (
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/pattern_matcher.py", line 96, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from .lowering import fallback_node_due_to_unsupported_type
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/lowering.py", line 6430, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from . import kernel
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/kernel/__init__.py", line 1, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from . import mm, mm_common, mm_plus_mm, unpack_mixed_mm
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/kernel/mm.py", line 16, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from torch._inductor.codegen.cpp_gemm_template import CppPackedGemmTemplate
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/codegen/cpp_gemm_template.py", line 19, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from .cpp_micro_gemm import CppMicroGemmAMX, create_micro_gemm, LayoutType
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/codegen/cpp_micro_gemm.py", line 16, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from .cpp_template_kernel import CppTemplateKernel
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/codegen/cpp_template_kernel.py", line 20, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from .cpp_wrapper_cpu import CppWrapperCpu
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/codegen/cpp_wrapper_cpu.py", line 30, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from .wrapper import EnterSubgraphLine, ExitSubgraphLine, WrapperCodeGen
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/codegen/wrapper.py", line 46, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from ..runtime import triton_heuristics
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_inductor/runtime/triton_heuristics.py", line 55, in <module>
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435]     from triton import Config
(VllmWorker rank=0 pid=565) ERROR 05-19 02:31:38 [multiproc_executor.py:435] ImportError: cannot import name 'Config' from 'triton' (unknown location)
(VllmWorker rank=0 pid=565) /vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py:442: ResourceWarning: Unclosed socket <zmq.Socket(zmq.SUB) at 0xffff59d13400>
(VllmWorker rank=0 pid=565)   shutdown_requested = True
(VllmWorker rank=0 pid=565) /vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py:442: ResourceWarning: Unclosed context <zmq.Context() at 0xffff953a2cf0>
(VllmWorker rank=0 pid=565)   shutdown_requested = True
(VllmWorker rank=0 pid=565) /vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py:442: ResourceWarning: Unclosed socket <zmq.Socket(zmq.XPUB) at 0xffff59d13580>
(VllmWorker rank=0 pid=565)   shutdown_requested = True
(VllmWorker rank=0 pid=565) /vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py:442: ResourceWarning: Unclosed context <zmq.Context() at 0xffff5b857a60>
(VllmWorker rank=0 pid=565)   shutdown_requested = True
/usr/local/python3.10.17/lib/python3.10/tempfile.py:869: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpe1ij7faf'>
  _warnings.warn(warn_message, ResourceWarning)
/usr/local/python3.10.17/lib/python3.10/tempfile.py:869: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpgiv5b5rs'>
  _warnings.warn(warn_message, ResourceWarning)
(VllmWorker rank=1 pid=566) /usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py:-1: ResourceWarning: Unclosed socket <zmq.Socket(zmq.SUB) at 0xffff63cbece0>
(VllmWorker rank=1 pid=566) /usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py:-1: ResourceWarning: Unclosed context <zmq.Context() at 0xffff65953790>
(VllmWorker rank=1 pid=566) /usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py:-1: ResourceWarning: Unclosed socket <zmq.Socket(zmq.XPUB) at 0xffff63cbee60>
(VllmWorker rank=1 pid=566) /usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py:-1: ResourceWarning: Unclosed context <zmq.Context() at 0xffff657fb920>
ERROR 05-19 02:31:42 [core.py:396] EngineCore failed to start.
ERROR 05-19 02:31:42 [core.py:396] Traceback (most recent call last):
ERROR 05-19 02:31:42 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 387, in run_engine_core
ERROR 05-19 02:31:42 [core.py:396]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 05-19 02:31:42 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 329, in __init__
ERROR 05-19 02:31:42 [core.py:396]     super().__init__(vllm_config, executor_class, log_stats,
ERROR 05-19 02:31:42 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 64, in __init__
ERROR 05-19 02:31:42 [core.py:396]     self.model_executor = executor_class(vllm_config)
ERROR 05-19 02:31:42 [core.py:396]   File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 52, in __init__
ERROR 05-19 02:31:42 [core.py:396]     self._init_executor()
ERROR 05-19 02:31:42 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 91, in _init_executor
ERROR 05-19 02:31:42 [core.py:396]     self.workers = WorkerProc.wait_for_ready(unready_workers)
ERROR 05-19 02:31:42 [core.py:396]   File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 370, in wait_for_ready
ERROR 05-19 02:31:42 [core.py:396]     raise e from None
ERROR 05-19 02:31:42 [core.py:396] Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
Process EngineCore_0:
Traceback (most recent call last):
  File "/usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 400, in run_engine_core
    raise e
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 387, in run_engine_core
    engine_core = EngineCoreProc(*args, **kwargs)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 329, in __init__
    super().__init__(vllm_config, executor_class, log_stats,
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 64, in __init__
    self.model_executor = executor_class(vllm_config)
  File "/vllm-workspace/vllm/vllm/executor/executor_base.py", line 52, in __init__
    self._init_executor()
  File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 91, in _init_executor
    self.workers = WorkerProc.wait_for_ready(unready_workers)
  File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 370, in wait_for_ready
    raise e from None
Exception: WorkerProc initialization failed due to an exception in a background process. See stack trace for root cause.
Traceback (most recent call last):
  File "/usr/local/python3.10.17/lib/python3.10/weakref.py", line 667, in _exitfunc
    f()
  File "/usr/local/python3.10.17/lib/python3.10/weakref.py", line 591, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py", line 228, in shutdown
    for w in self.workers:
AttributeError: 'MultiprocExecutor' object has no attribute 'workers'
/vllm-workspace/vllm/vllm/v1/executor/multiproc_executor.py:232: ResourceWarning: Unclosed socket <zmq.Socket(zmq.XPUB) at 0xfffd413a63e0>
  self.rpc_broadcast_mq = None
/usr/local/python3.10.17/lib/python3.10/tempfile.py:869: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmp4774i3gq'>
  _warnings.warn(warn_message, ResourceWarning)
Traceback (most recent call last):
  File "/usr/local/python3.10.17/bin/vllm", line 8, in <module>
    sys.exit(main())
  File "/vllm-workspace/vllm/vllm/entrypoints/cli/main.py", line 54, in main
    args.dispatch_function(args)
  File "/vllm-workspace/vllm/vllm/entrypoints/cli/serve.py", line 27, in cmd
    uvloop.run(run_server(args))
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1078, in run_server
    async with build_async_engine_client(args) as engine_client:
  File "/usr/local/python3.10.17/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 146, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
  File "/usr/local/python3.10.17/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 178, in build_async_engine_client_from_engine_args
    async_llm = AsyncLLM.from_vllm_config(
  File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 150, in from_vllm_config
    return cls(
  File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 118, in __init__
    self.engine_core = core_client_class(
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 642, in __init__
    super().__init__(
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 398, in __init__
    self._wait_for_engine_startup()
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 430, in _wait_for_engine_startup
    raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above.
[ERROR] 2025-05-19-02:31:45 (PID:155, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
/usr/local/python3.10.17/lib/python3.10/tempfile.py:869: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpgzh3ehf2'>
  _warnings.warn(warn_message, ResourceWarning)
sys:1: ResourceWarning: unclosed <socket.socket fd=15, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 8000)>
root@a8eadd4a7ca2:/data1/scripts/vllm_ascend_qwen3# /usr/local/python3.10.17/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

[dawnranger]

安装失败

[dawnranger]

vllm==0.8.5.post1 要求的torch版本是2.6.0，但是vllm-ascend=0.8.5rc1 的torch版本是2.5.1，会发生冲突

[wangxiyuan]

@dawnranger torch version conflict doesn't affect the real work. The error rasied can be ignored.

[wangxiyuan]

安装失败

it seems that there is problem in CANN