[v0.9.1rc2] FAQ / Feedback | 问题/反馈

[Yikun]

Please use doc: https://vllm-ascend.readthedocs.io/en/latest/installation.html

#1486

---

请使用 https://vllm-ascend.readthedocs.io/en/latest/installation.html 安装

Quickstart: https://vllm-ascend.readthedocs.io/en/latest/quick_start.html
Atlas 300I series: https://vllm-ascend.readthedocs.io/en/latest/tutorials/single_node_300i.html
Accuracy report: https://vllm-ascend.readthedocs.io/en/latest/developer_guide/evaluation/accuracy_report/index.html

[AlphaINF]

在300I上，使用vllm-ascend运行qwen3-8b吞吐量很低，仅能达到2token/s,而在mindie上可以达到10token/s
我试图关闭enforce-eager，结果会在cuda-graph阶段提示存在有错误，如下所示：

我的启动命令如下:

# 设置镜像版本
export IMAGE=quay.io/ascend/vllm-ascend:v0.9.1rc1-310p

# 启动容器并在后台运行vLLM服务
docker run --rm \
        --name qwen3-8b \
        --device /dev/davinci0 \
        --device /dev/davinci_manager \
        --device /dev/devmm_svm \
        --device /dev/hisi_hdc \
        -v /usr/local/dcmi:/usr/local/dcmi \
        -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
        -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
        -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
        -v /etc/ascend_install.info:/etc/ascend_install.info \
        -v /root/.cache:/root/.cache \
        -v /home/HwHiAiUser/models:/models \
        -v /root/myb/vllm-ascend:/workspace \
        -v /root/myb/mindie2_test/Qwen3-8B/qwen8b_wxey:/root/myb/mindie2_test/Qwen3-8B/qwen8b_wxey \
        -e OMP_NUM_THREADS=48 \
        -e VLLM_USE_V1=1 \
        -p 8000:8000 \
        $IMAGE \
        vllm serve --model="/models/Qwen3-8B" \
            --host 0.0.0.0 \
            --port 8000 \
            --max-num-seqs 32 \
            --max-model-len 32768 \
            --served-model-name "qwen8b" \
            --dtype float16 \
            --disable-custom-all-reduce \
            --compilation-config '{"custom_ops":["+rms_norm", "+rotary_embedding"]}'

# echo "vLLM服务已启动，可以通过以下命令测试："
# echo "curl http://localhost:8000/v1/completions \\"
# echo "-H \"Content-Type: application/json\" \\"
# echo "-d '{\"model\": \"/models/Qwen3-8B\", \"prompt\": \"The future of AI is\", \"max_tokens\": 50}'"

INFO 06-29 06:15:40 [importing.py:17] Triton not installed or not compatible; certain GPU-related functions will not be available.
WARNING 06-29 06:15:40 [importing.py:29] Triton is not installed. Using dummy decorators. Install it via `pip install triton` to enable kernel compilation.
INFO 06-29 06:15:41 [__init__.py:39] Available plugins for group vllm.platform_plugins:
INFO 06-29 06:15:41 [__init__.py:41] - ascend -> vllm_ascend:register
INFO 06-29 06:15:41 [__init__.py:44] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 06-29 06:15:41 [__init__.py:235] Platform plugin ascend is activated
WARNING 06-29 06:15:45 [_custom_ops.py:22] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'")
WARNING 06-29 06:15:50 [registry.py:401] Model architecture DeepSeekMTPModel is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_mtp:CustomDeepSeekMTP.
WARNING 06-29 06:15:50 [registry.py:401] Model architecture Qwen2VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_vl:AscendQwen2VLForConditionalGeneration.
WARNING 06-29 06:15:50 [registry.py:401] Model architecture Qwen2_5_VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl:AscendQwen2_5_VLForConditionalGeneration.
WARNING 06-29 06:15:50 [registry.py:401] Model architecture DeepseekV2ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v2:CustomDeepseekV2ForCausalLM.
WARNING 06-29 06:15:50 [registry.py:401] Model architecture DeepseekV3ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v2:CustomDeepseekV3ForCausalLM.
WARNING 06-29 06:15:50 [registry.py:401] Model architecture Qwen3MoeForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen3_moe:CustomQwen3MoeForCausalLM.
INFO 06-29 06:15:51 [config.py:1980] Disabled the custom all-reduce kernel because it is not supported on current platform.
INFO 06-29 06:15:53 [config.py:1980] Disabled the custom all-reduce kernel because it is not supported on current platform.
INFO 06-29 06:15:55 [config.py:1980] Disabled the custom all-reduce kernel because it is not supported on current platform.
INFO 06-29 06:15:56 [config.py:1980] Disabled the custom all-reduce kernel because it is not supported on current platform.
INFO 06-29 06:15:56 [api_server.py:1287] vLLM API server version 0.9.1
INFO 06-29 06:15:57 [config.py:1980] Disabled the custom all-reduce kernel because it is not supported on current platform.
INFO 06-29 06:15:57 [cli_args.py:309] non-default args: {'host': '0.0.0.0', 'model': '/models/Qwen3-8B', 'dtype': 'float16', 'max_model_len': 32768, 'served_model_name': ['qwen8b'], 'disable_custom_all_reduce': True, 'max_num_seqs': 32, 'compilation_config': {"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":["+rms_norm","+rotary_embedding"],"splitting_ops":[],"use_inductor":true,"compile_sizes":null,"inductor_compile_config":{},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":null,"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":null,"local_cache_dir":null}}
INFO 06-29 06:16:14 [config.py:823] This model supports multiple tasks: {'embed', 'classify', 'generate', 'score', 'reward'}. Defaulting to 'generate'.
WARNING 06-29 06:16:14 [arg_utils.py:1647] Detected VLLM_USE_V1=1 with npu. Usage should be considered experimental. Please report any issues on Github.
INFO 06-29 06:16:14 [config.py:1980] Disabled the custom all-reduce kernel because it is not supported on current platform.
INFO 06-29 06:16:14 [config.py:2195] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 06-29 06:16:14 [platform.py:177] PIECEWISE compilation enabled on NPU. use_inductor not supported - using only ACL Graph mode
INFO 06-29 06:16:14 [utils.py:297] Calculated maximum supported batch sizes for ACL graph: 51
INFO 06-29 06:16:14 [utils.py:312] Adjusted ACL graph batch sizes for Qwen3ForCausalLM model (layers: 36): 67 → 51 sizes
WARNING 06-29 06:16:19 [env_override.py:17] NCCL_CUMEM_ENABLE is set to 0, skipping override. This may increase memory overhead with cudagraph+allreduce: https://github.com/NVIDIA/nccl/issues/1234
INFO 06-29 06:16:21 [importing.py:17] Triton not installed or not compatible; certain GPU-related functions will not be available.
WARNING 06-29 06:16:21 [importing.py:29] Triton is not installed. Using dummy decorators. Install it via `pip install triton` to enable kernel compilation.
INFO 06-29 06:16:22 [__init__.py:39] Available plugins for group vllm.platform_plugins:
INFO 06-29 06:16:22 [__init__.py:41] - ascend -> vllm_ascend:register
INFO 06-29 06:16:22 [__init__.py:44] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 06-29 06:16:22 [__init__.py:235] Platform plugin ascend is activated
WARNING 06-29 06:16:26 [_custom_ops.py:22] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'")
INFO 06-29 06:16:28 [core.py:455] Waiting for init message from front-end.
INFO 06-29 06:16:28 [platform.py:177] PIECEWISE compilation enabled on NPU. use_inductor not supported - using only ACL Graph mode
INFO 06-29 06:16:28 [utils.py:297] Calculated maximum supported batch sizes for ACL graph: 51
INFO 06-29 06:16:28 [utils.py:323] No adjustment needed for ACL graph batch sizes: Qwen3ForCausalLM model (layers: 36) with 51 sizes
WARNING 06-29 06:16:30 [registry.py:401] Model architecture DeepSeekMTPModel is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_mtp:CustomDeepSeekMTP.
WARNING 06-29 06:16:30 [registry.py:401] Model architecture Qwen2VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_vl:AscendQwen2VLForConditionalGeneration.
WARNING 06-29 06:16:30 [registry.py:401] Model architecture Qwen2_5_VLForConditionalGeneration is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen2_5_vl:AscendQwen2_5_VLForConditionalGeneration.
WARNING 06-29 06:16:30 [registry.py:401] Model architecture DeepseekV2ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v2:CustomDeepseekV2ForCausalLM.
WARNING 06-29 06:16:30 [registry.py:401] Model architecture DeepseekV3ForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.deepseek_v2:CustomDeepseekV3ForCausalLM.
WARNING 06-29 06:16:30 [registry.py:401] Model architecture Qwen3MoeForCausalLM is already registered, and will be overwritten by the new model class vllm_ascend.models.qwen3_moe:CustomQwen3MoeForCausalLM.
INFO 06-29 06:16:30 [core.py:70] Initializing a V1 LLM engine (v0.9.1) with config: model='/models/Qwen3-8B', speculative_config=None, tokenizer='/models/Qwen3-8B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=32768, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=False, kv_cache_dtype=auto,  device_config=npu, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=qwen8b, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":["+rms_norm","+rotary_embedding"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output","vllm.unified_ascend_attention_with_output","vllm.unified_ascend_attention_with_output"],"use_inductor":false,"compile_sizes":[],"inductor_compile_config":{},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,488,480,472,456,448,440,424,416,408,392,384,376,368,352,344,336,320,312,304,288,280,272,256,248,240,224,216,208,192,184,176,160,152,144,128,120,112,104,88,80,72,56,48,40,24,16,8,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":512,"local_cache_dir":null}
WARNING 06-29 06:16:30 [utils.py:2737] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes not implemented in <vllm_ascend.worker.worker_v1.NPUWorker object at 0xfffe967d6110>
WARNING 06-29 06:16:39 [env_override.py:17] NCCL_CUMEM_ENABLE is set to 0, skipping override. This may increase memory overhead with cudagraph+allreduce: https://github.com/NVIDIA/nccl/issues/1234
INFO 06-29 06:16:40 [importing.py:17] Triton not installed or not compatible; certain GPU-related functions will not be available.
WARNING 06-29 06:16:40 [importing.py:29] Triton is not installed. Using dummy decorators. Install it via `pip install triton` to enable kernel compilation.
INFO 06-29 06:16:41 [__init__.py:39] Available plugins for group vllm.platform_plugins:
INFO 06-29 06:16:41 [__init__.py:41] - ascend -> vllm_ascend:register
INFO 06-29 06:16:41 [__init__.py:44] All plugins in this group will be loaded. Set `VLLM_PLUGINS` to control which plugins to load.
INFO 06-29 06:16:41 [__init__.py:235] Platform plugin ascend is activated
WARNING 06-29 06:16:45 [_custom_ops.py:22] Failed to import from vllm._C with ModuleNotFoundError("No module named 'vllm._C'")
INFO 06-29 06:16:49 [parallel_state.py:1065] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
INFO 06-29 06:17:08 [model_runner_v1.py:1829] Starting to load model /models/Qwen3-8B...
Loading safetensors checkpoint shards:   0% Completed | 0/5 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:  20% Completed | 1/5 [00:01<00:04,  1.20s/it]
Loading safetensors checkpoint shards:  40% Completed | 2/5 [00:02<00:04,  1.35s/it]
Loading safetensors checkpoint shards:  60% Completed | 3/5 [00:04<00:02,  1.40s/it]
Loading safetensors checkpoint shards:  80% Completed | 4/5 [00:04<00:01,  1.08s/it]
Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:05<00:00,  1.08s/it]
Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:05<00:00,  1.16s/it]

INFO 06-29 06:17:42 [default_loader.py:272] Loading weights took 5.95 seconds
INFO 06-29 06:17:42 [model_runner_v1.py:1848] Loading model weights took 15.2684 GB
INFO 06-29 06:17:54 [backends.py:462] Using cache directory: /root/.cache/vllm/torch_compile_cache/9d36d822fa/rank_0_0 for vLLM's torch.compile
INFO 06-29 06:17:54 [backends.py:472] Dynamo bytecode transform time: 11.26 s
INFO 06-29 06:17:59 [backends.py:173] Compiling a graph for general shape takes 3.57 s
.....INFO 06-29 06:18:46 [monitor.py:34] torch.compile takes 14.83 s in total
..INFO 06-29 06:19:12 [kv_cache_utils.py:715] GPU KV cache size: 160,000 tokens
INFO 06-29 06:19:12 [kv_cache_utils.py:719] Maximum concurrency for 32,768 tokens per request: 4.88x
.....ERROR 06-29 06:20:03 [core.py:515] EngineCore failed to start.
ERROR 06-29 06:20:03 [core.py:515] Traceback (most recent call last):
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 506, in run_engine_core
ERROR 06-29 06:20:03 [core.py:515]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 390, in __init__
ERROR 06-29 06:20:03 [core.py:515]     super().__init__(vllm_config, executor_class, log_stats,
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 83, in __init__
ERROR 06-29 06:20:03 [core.py:515]     self._initialize_kv_caches(vllm_config)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 168, in _initialize_kv_caches
ERROR 06-29 06:20:03 [core.py:515]     self.model_executor.initialize_from_config(kv_cache_configs)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/v1/executor/abstract.py", line 66, in initialize_from_config
ERROR 06-29 06:20:03 [core.py:515]     self.collective_rpc("compile_or_warm_up_model")
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
ERROR 06-29 06:20:03 [core.py:515]     answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/utils.py", line 2671, in run_method
ERROR 06-29 06:20:03 [core.py:515]     return func(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 205, in compile_or_warm_up_model
ERROR 06-29 06:20:03 [core.py:515]     self.model_runner.capture_model()
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 2072, in capture_model
ERROR 06-29 06:20:03 [core.py:515]     self._dummy_run(num_tokens)
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 06-29 06:20:03 [core.py:515]     return func(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 1774, in _dummy_run
ERROR 06-29 06:20:03 [core.py:515]     hidden_states = model(
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 06-29 06:20:03 [core.py:515]     return self._call_impl(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 06-29 06:20:03 [core.py:515]     return forward_call(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen3.py", line 301, in forward
ERROR 06-29 06:20:03 [core.py:515]     hidden_states = self.model(input_ids, positions, intermediate_tensors,
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/compilation/decorators.py", line 246, in __call__
ERROR 06-29 06:20:03 [core.py:515]     model_output = self.forward(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm/vllm/model_executor/models/qwen2.py", line 336, in forward
ERROR 06-29 06:20:03 [core.py:515]     def forward(
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 06-29 06:20:03 [core.py:515]     return self._call_impl(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 06-29 06:20:03 [core.py:515]     return forward_call(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 632, in _fn
ERROR 06-29 06:20:03 [core.py:515]     return fn(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/fx/graph_module.py", line 784, in call_wrapped
ERROR 06-29 06:20:03 [core.py:515]     return self._wrapped_call(self, *args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/fx/graph_module.py", line 361, in __call__
ERROR 06-29 06:20:03 [core.py:515]     raise e
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/fx/graph_module.py", line 348, in __call__
ERROR 06-29 06:20:03 [core.py:515]     return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 06-29 06:20:03 [core.py:515]     return self._call_impl(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 06-29 06:20:03 [core.py:515]     return forward_call(*args, **kwargs)
ERROR 06-29 06:20:03 [core.py:515]   File "<eval_with_key>.74", line 298, in forward
ERROR 06-29 06:20:03 [core.py:515]     submod_0 = self.submod_0(l_input_ids_, s0, l_self_modules_embed_tokens_parameters_weight_, l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_q_norm_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_k_norm_parameters_weight_, l_positions_, s1, l_self_modules_layers_modules_0_modules_self_attn_modules_rotary_emb_buffers_cos_sin_cache_);  l_input_ids_ = l_self_modules_embed_tokens_parameters_weight_ = l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_q_norm_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_k_norm_parameters_weight_ = None
ERROR 06-29 06:20:03 [core.py:515]   File "/vllm-workspace/vllm-ascend/vllm_ascend/compilation/piecewise_backend.py", line 192, in __call__
ERROR 06-29 06:20:03 [core.py:515]     with torch.npu.graph(aclgraph, pool=self.graph_pool):
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch_npu/npu/graphs.py", line 291, in __enter__
ERROR 06-29 06:20:03 [core.py:515]     self.npu_graph.capture_begin(
ERROR 06-29 06:20:03 [core.py:515]   File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch_npu/npu/graphs.py", line 191, in capture_begin
ERROR 06-29 06:20:03 [core.py:515]     super().capture_begin(pool=pool, capture_error_mode=capture_error_mode)
ERROR 06-29 06:20:03 [core.py:515] RuntimeError: capture_begin:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:158 NPU function error: c10_npu::acl::AclmdlRICaptureBegin(capture_stream_, capture_mode), error code is 207000
ERROR 06-29 06:20:03 [core.py:515] [ERROR] 2025-06-29-06:20:03 (PID:230, Device:0, RankID:-1) ERR00100 PTA call acl api failed
ERROR 06-29 06:20:03 [core.py:515] [Error]: This feature is not supported.
ERROR 06-29 06:20:03 [core.py:515]         Rectify the fault based on the error information in the ascend log.
ERROR 06-29 06:20:03 [core.py:515] EH9999: Inner Error!
ERROR 06-29 06:20:03 [core.py:515] EH9999: [PID: 230] 2025-06-29-06:20:03.334.603 begin capture stream failed, runtime result = 207000[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
ERROR 06-29 06:20:03 [core.py:515]         TraceBack (most recent call last):
ERROR 06-29 06:20:03 [core.py:515] 
Process EngineCore_0:
Traceback (most recent call last):
  File "/usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/python3.10.17/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 519, in run_engine_core
    raise e
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 506, in run_engine_core
    engine_core = EngineCoreProc(*args, **kwargs)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 390, in __init__
    super().__init__(vllm_config, executor_class, log_stats,
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 83, in __init__
    self._initialize_kv_caches(vllm_config)
  File "/vllm-workspace/vllm/vllm/v1/engine/core.py", line 168, in _initialize_kv_caches
    self.model_executor.initialize_from_config(kv_cache_configs)
  File "/vllm-workspace/vllm/vllm/v1/executor/abstract.py", line 66, in initialize_from_config
    self.collective_rpc("compile_or_warm_up_model")
  File "/vllm-workspace/vllm/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
    answer = run_method(self.driver_worker, method, args, kwargs)
  File "/vllm-workspace/vllm/vllm/utils.py", line 2671, in run_method
    return func(*args, **kwargs)
  File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/worker_v1.py", line 205, in compile_or_warm_up_model
    self.model_runner.capture_model()
  File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 2072, in capture_model
    self._dummy_run(num_tokens)
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/vllm-workspace/vllm-ascend/vllm_ascend/worker/model_runner_v1.py", line 1774, in _dummy_run
    hidden_states = model(
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/vllm-workspace/vllm/vllm/model_executor/models/qwen3.py", line 301, in forward
    hidden_states = self.model(input_ids, positions, intermediate_tensors,
  File "/vllm-workspace/vllm/vllm/compilation/decorators.py", line 246, in __call__
    model_output = self.forward(*args, **kwargs)
  File "/vllm-workspace/vllm/vllm/model_executor/models/qwen2.py", line 336, in forward
    def forward(
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 632, in _fn
    return fn(*args, **kwargs)
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/fx/graph_module.py", line 784, in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/fx/graph_module.py", line 361, in __call__
    raise e
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/fx/graph_module.py", line 348, in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "<eval_with_key>.74", line 298, in forward
    submod_0 = self.submod_0(l_input_ids_, s0, l_self_modules_embed_tokens_parameters_weight_, l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_q_norm_parameters_weight_, l_self_modules_layers_modules_0_modules_self_attn_modules_k_norm_parameters_weight_, l_positions_, s1, l_self_modules_layers_modules_0_modules_self_attn_modules_rotary_emb_buffers_cos_sin_cache_);  l_input_ids_ = l_self_modules_embed_tokens_parameters_weight_ = l_self_modules_layers_modules_0_modules_input_layernorm_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_qkv_proj_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_q_norm_parameters_weight_ = l_self_modules_layers_modules_0_modules_self_attn_modules_k_norm_parameters_weight_ = None
  File "/vllm-workspace/vllm-ascend/vllm_ascend/compilation/piecewise_backend.py", line 192, in __call__
    with torch.npu.graph(aclgraph, pool=self.graph_pool):
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch_npu/npu/graphs.py", line 291, in __enter__
    self.npu_graph.capture_begin(
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/torch_npu/npu/graphs.py", line 191, in capture_begin
    super().capture_begin(pool=pool, capture_error_mode=capture_error_mode)
RuntimeError: capture_begin:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:158 NPU function error: c10_npu::acl::AclmdlRICaptureBegin(capture_stream_, capture_mode), error code is 207000
[ERROR] 2025-06-29-06:20:03 (PID:230, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: This feature is not supported.
        Rectify the fault based on the error information in the ascend log.
EH9999: Inner Error!
EH9999: [PID: 230] 2025-06-29-06:20:03.334.603 begin capture stream failed, runtime result = 207000[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
        TraceBack (most recent call last):

Traceback (most recent call last):
  File "/usr/local/python3.10.17/bin/vllm", line 8, in <module>
    sys.exit(main())
  File "/vllm-workspace/vllm/vllm/entrypoints/cli/main.py", line 59, in main
    args.dispatch_function(args)
  File "/vllm-workspace/vllm/vllm/entrypoints/cli/serve.py", line 58, in cmd
    uvloop.run(run_server(args))
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/python3.10.17/lib/python3.10/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1323, in run_server
    await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 1343, in run_server_worker
    async with build_async_engine_client(args, client_config) as engine_client:
  File "/usr/local/python3.10.17/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 155, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
  File "/usr/local/python3.10.17/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/vllm-workspace/vllm/vllm/entrypoints/openai/api_server.py", line 191, in build_async_engine_client_from_engine_args
    async_llm = AsyncLLM.from_vllm_config(
  File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 162, in from_vllm_config
    return cls(
  File "/vllm-workspace/vllm/vllm/v1/engine/async_llm.py", line 124, in __init__
    self.engine_core = EngineCoreClient.make_async_mp_client(
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 93, in make_async_mp_client
    return AsyncMPClient(vllm_config, executor_class, log_stats,
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 716, in __init__
    super().__init__(
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 422, in __init__
    self._init_engines_direct(vllm_config, local_only,
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 491, in _init_engines_direct
    self._wait_for_engine_startup(handshake_socket, input_address,
  File "/vllm-workspace/vllm/vllm/v1/engine/core_client.py", line 511, in _wait_for_engine_startup
    wait_for_engine_startup(
  File "/vllm-workspace/vllm/vllm/v1/utils.py", line 494, in wait_for_engine_startup
    raise RuntimeError("Engine core initialization failed. "
RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}
[ERROR] 2025-06-29-06:20:16 (PID:1, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception

[Yikun]

@AlphaINF For Altlas 300I series, pls try: https://vllm-ascend.readthedocs.io/en/latest/tutorials/single_node_300i.html

[icy-fish]

Hi, for dependency torch-npu, I noticed that it had been updated according to this PR #1347 .

And the pypi package of torch-npu with specified version can only be found in HuaweiCloud mirror: https://mirrors.huaweicloud.com/ascend/repos/pypi , but not in official Pypi registry: https://pypi.org/project/torch-npu/#history

Why specified torch-npu package not pushed to official Pypi registry? And what's the plan for future?

[Yikun]

@icy-fish https://vllm-ascend.readthedocs.io/en/latest/community/versioning_policy.html#software-dependency-management

See here, It's the POC version, and it will bee replaced once final release pypi 2.5.1.post1 is released.

[Yikun]

Thanks for all your feedback! The release has been replaced with latest one v0.9.1 version:

• Please try new one: https://vllm-ascend.readthedocs.io/en/v0.9.1-dev/quick_start.html
• Please try recommand version: https://github.com/vllm-project/vllm-ascend?tab=readme-ov-file#getting-started