Skip to content

Commit b91f9a5

Browse files
authored
[Doc][Build] Update build doc and faq (#568)
Update build doc and faq about deepseek w8a8 Signed-off-by: MengqingCao <cmq0113@163.com>
1 parent e66ded5 commit b91f9a5

File tree

2 files changed

+3
-5
lines changed

2 files changed

+3
-5
lines changed

docs/source/faqs.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -84,11 +84,9 @@ Currently, there is no quantization method supported in vllm-ascend originally.
8484

8585
### 11. How to run w8a8 DeepSeek model?
8686

87-
Currently, running on v0.7.3, we should run w8a8 with vllm + vllm-ascend + mindie-turbo. And we only need vllm + vllm-ascend when v0.8.X is released. After installing the above packages, you can follow the steps below to run w8a8 DeepSeek:
87+
Currently, w8a8 DeepSeek is working in process: [support AscendW8A8 quantization](https://github.com/vllm-project/vllm-ascend/pull/511)
8888

89-
1. Quantize bf16 DeepSeek, e.g. [unsloth/DeepSeek-R1-BF16](https://modelscope.cn/models/unsloth/DeepSeek-R1-BF16), with msModelSlim to get w8a8 DeepSeek. Find more details in [msModelSlim doc](https://gitee.com/ascend/msit/tree/master/msmodelslim/msmodelslim/pytorch/llm_ptq)
90-
2. Copy the content of `quant_model_description_w8a8_dynamic.json` into the `quantization_config` of `config.json` of the quantized model files.
91-
3. Reference with the quantized DeepSeek model.
89+
Please run DeepSeek with BF16 now, follwing the [Multi-Node DeepSeek inferencing tutorail](https://vllm-ascend.readthedocs.io/en/main/tutorials/multi_node.html)
9290

9391
### 12. There is not output in log when loading models using vllm-ascend, How to solve it?
9492

docs/source/installation.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ python -m venv vllm-ascend-env
7878
source vllm-ascend-env/bin/activate
7979

8080
# Install required python packages.
81-
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple attrs numpy<2.0.0 decorator sympy cffi pyyaml pathlib2 psutil protobuf scipy requests absl-py wheel typing_extensions
81+
pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple attrs 'numpy<2.0.0' decorator sympy cffi pyyaml pathlib2 psutil protobuf scipy requests absl-py wheel typing_extensions
8282

8383
# Download and install the CANN package.
8484
wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/CANN/CANN%208.0.0/Ascend-cann-toolkit_8.0.0_linux-aarch64.run

0 commit comments

Comments
 (0)