Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,11 @@ By using vLLM Ascend plugin, popular open-source models, including Transformer-l
## Prerequisites

- Hardware: Atlas 800I A2 Inference series, Atlas A2 Training series
- Software: vLLM (the same version as vllm-ascned), Python >= 3.9, CANN >= 8.0.RC2, PyTorch >= 2.4.0, torch-npu >= 2.4.0
- Software:
* Python >= 3.9
* CANN >= 8.0.RC2
* PyTorch >= 2.4.0, torch-npu >= 2.4.0
* vLLM (the same version as vllm-ascend)

Find more about how to setup your environment step by step in [here](docs/installation.md).

Expand Down Expand Up @@ -64,7 +68,7 @@ Run the following command to start the vLLM server with the [Qwen/Qwen2.5-0.5B-I
vllm serve Qwen/Qwen2.5-0.5B-Instruct
curl http://localhost:8000/v1/models
```
**Please refer to [Official Docs](./docs/index.md) for more details.**
**Please refer to [official docs](./docs/index.md) for more details.**

## Contributing
See [CONTRIBUTING](./CONTRIBUTING.md) for more details, which is a step-by-step guide to help you set up development environment, build and test.
Expand Down
8 changes: 6 additions & 2 deletions README.zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,13 @@ vLLM 昇腾插件 (`vllm-ascend`) 是一个让vLLM在Ascend NPU无缝运行的
## 准备

- 硬件:Atlas 800I A2 Inference系列、Atlas A2 Training系列
- 软件:vLLM(与vllm-ascn​​ed版本相同),Python >= 3.9,CANN >= 8.0.RC2,PyTorch >= 2.4.0,torch-npu >= 2.4.0
- 软件:
* Python >= 3.9
* CANN >= 8.0.RC2
* PyTorch >= 2.4.0, torch-npu >= 2.4.0
* vLLM (与vllm-ascend版本一致)

在[此处](docs/installation.md) 中查找有关如何逐步设置环境的更多信息
在[此处](docs/installation.md),您可以了解如何逐步准备环境

## 开始使用

Expand Down
39 changes: 18 additions & 21 deletions docs/installation.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,14 @@
# Installation


## Building

#### Build Python package from source

```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
pip install -e .
```

#### Build container image from source
```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
docker build -t vllm-ascend-dev-image -f ./Dockerfile .
```

### Prepare Ascend NPU environment

### Dependencies
### 1. Dependencies
| Requirement | Supported version | Recommended version | Note |
| ------------ | ------- | ----------- | ----------- |
| Python | >= 3.9 | [3.10](https://www.python.org/downloads/) | Required for vllm |
| CANN | >= 8.0.RC2 | [8.0.RC3](https://www.hiascend.com/developer/download/community/result?module=cann&cann=8.0.0.beta1) | Required for vllm-ascend and torch-npu |
| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | Required for vllm-ascend |
| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | Required for torch-npu and vllm required |

### 2. Prepare Ascend NPU environment

Below is a quick note to install recommended version software:

Expand Down Expand Up @@ -56,3 +37,19 @@ You do not need to install `torch` and `torch_npu` manually, they will be automa

Or follow the instructions provided in the [Ascend Installation Guide](https://ascend.github.io/docs/sources/ascend/quick_install.html) to set up the environment.

### 3. Building

#### Build Python package from source

```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
pip install -e .
```

#### Build container image from source
```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
docker build -t vllm-ascend-dev-image -f ./Dockerfile .
```