Skip to content

Commit d5e7756

Browse files
YikunwangxiyuanMengqingCaowangshuai09shen-shanshan
authored
[Core] Init vllm-ascend (#3)
### What this PR does / why we need it? vLLM Ascend plugin (vllm-ascend) is a backend plugin for running vLLM on the Ascend NPU. This plugin is the recommended approach for supporting the Ascend backend within the vLLM community. It adheres to the principles outlined in the [RFC]: Hardware pluggable, providing a hardware-pluggable interface that decouples the integration of the Ascend NPU with vLLM. This patch also include changes to make CI work and use cache speed up e2e test, including: 1. Change push (post merge ci) and pull_request (pr ci) trigger branch to main 2. Make mypy work by ignore base_communicator and clear unused deps 3. Several improvements for vllm_ascend_test: - use cache (pip, ms, hf) speed up e2e test (25mins --> 5mins) - switch `git clone` command to `action/checkout` to speedup checkout and - Enable sv for pytest for better info dump - Remove network host to resole `docker: conflicting ontions: cannot attach both user-defined and non-user-definednetwork-modes`, which is a problem on docker 1.45 but not on 1.39. 4. Adapt MLA decode optimizations: vllm-project/vllm@cabaf4e ### Does this PR introduce _any_ user-facing change? Yes, init the PR. ### How was this patch tested? - This is the first PR to make ascend NPU work on vLLM. All code is tested on ascend with vLLM V0 Engine. - CI passed --------- Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: Yikun Jiang <yikunkero@gmail.com> Co-authored-by: wangxiyuan <wangxiyuan1007@gmail.com> Co-authored-by: MengqingCao <cmq0113@163.com> Co-authored-by: wangshuai09 <391746016@qq.com> Co-authored-by: Shanshan Shen <467638484@qq.com> Co-authored-by: wangli <wangli858794774@gmail.com>
1 parent eb28342 commit d5e7756

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+5161
-0
lines changed

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
<!-- Thanks for sending a pull request!
2+
3+
BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing/overview.html
4+
5+
-->
6+
### What this PR does / why we need it?
7+
<!--
8+
- Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
9+
If possible, please consider writing useful notes for better and faster reviews in your PR.
10+
11+
- Please clarify why the changes are needed. For instance, the use case and bug description.
12+
13+
- Fixes #
14+
-->
15+
16+
### Does this PR introduce _any_ user-facing change?
17+
<!--
18+
Note that it means *any* user-facing change including all aspects such as API, interface or other behavior changes.
19+
Documentation-only updates are not considered user-facing changes.
20+
-->
21+
22+
### How was this patch tested?
23+
<!--
24+
CI passed with new added/existing test.
25+
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
26+
If tests were not added, please describe why they were not added and/or why it was difficult to add.
27+
-->
28+

‎.github/workflows/actionlint.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
#
2+
# Adapted from vllm-project/vllm/blob/main/.github
3+
# Copyright 2023 The vLLM team.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
name: Lint GitHub Actions workflows
19+
on:
20+
push:
21+
branches:
22+
- "main"
23+
paths:
24+
- '.github/workflows/*.ya?ml'
25+
- '.github/workflows/actionlint.*'
26+
- '.github/workflows/matchers/actionlint.json'
27+
pull_request:
28+
branches:
29+
- "main"
30+
paths:
31+
- '.github/workflows/*.ya?ml'
32+
- '.github/workflows/actionlint.*'
33+
- '.github/workflows/matchers/actionlint.json'
34+
35+
env:
36+
LC_ALL: en_US.UTF-8
37+
38+
defaults:
39+
run:
40+
shell: bash
41+
42+
permissions:
43+
contents: read
44+
45+
jobs:
46+
actionlint:
47+
runs-on: ubuntu-latest
48+
steps:
49+
- name: "Checkout"
50+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
51+
with:
52+
fetch-depth: 0
53+
54+
- name: "Run actionlint"
55+
run: |
56+
echo "::add-matcher::.github/workflows/matchers/actionlint.json"
57+
tools/actionlint.sh -color

‎.github/workflows/matchers/actionlint.json‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"owner": "actionlint",
5+
"pattern": [
6+
{
7+
"regexp": "^(?:\\x1b\\[\\d+m)?(.+?)(?:\\x1b\\[\\d+m)*:(?:\\x1b\\[\\d+m)*(\\d+)(?:\\x1b\\[\\d+m)*:(?:\\x1b\\[\\d+m)*(\\d+)(?:\\x1b\\[\\d+m)*: (?:\\x1b\\[\\d+m)*(.+?)(?:\\x1b\\[\\d+m)* \\[(.+?)\\]$",
8+
"file": 1,
9+
"line": 2,
10+
"column": 3,
11+
"message": 4,
12+
"code": 5
13+
}
14+
]
15+
}
16+
]
17+
}

‎.github/workflows/matchers/mypy.json‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"owner": "mypy",
5+
"pattern": [
6+
{
7+
"regexp": "^(.+):(\\d+):\\s(error|warning):\\s(.+)$",
8+
"file": 1,
9+
"line": 2,
10+
"severity": 3,
11+
"message": 4
12+
}
13+
]
14+
}
15+
]
16+
}

‎.github/workflows/matchers/ruff.json‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"problemMatcher": [
3+
{
4+
"owner": "ruff",
5+
"pattern": [
6+
{
7+
"regexp": "^(.+?):(\\d+):(\\d+): (\\w+): (.+)$",
8+
"file": 1,
9+
"line": 2,
10+
"column": 3,
11+
"code": 4,
12+
"message": 5
13+
}
14+
]
15+
}
16+
]
17+
}

‎.github/workflows/mypy.yaml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
#
2+
# Adapted from vllm-project/vllm/blob/main/.github
3+
# Copyright 2023 The vLLM team.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
name: mypy
19+
20+
on:
21+
# Trigger the workflow on push or pull request,
22+
# but only for the main branch
23+
push:
24+
branches:
25+
- "main"
26+
paths:
27+
- '**/*.py'
28+
- '.github/workflows/mypy.yaml'
29+
- 'tools/mypy.sh'
30+
pull_request:
31+
branches:
32+
- "main"
33+
# This workflow is only relevant when one of the following files changes.
34+
# However, we have github configured to expect and require this workflow
35+
# to run and pass before github with auto-merge a pull request. Until github
36+
# allows more flexible auto-merge policy, we can just run this on every PR.
37+
# It doesn't take that long to run, anyway.
38+
paths:
39+
- '**/*.py'
40+
- '.github/workflows/mypy.yaml'
41+
- 'tools/mypy.sh'
42+
43+
jobs:
44+
mypy:
45+
runs-on: ubuntu-latest
46+
strategy:
47+
matrix:
48+
python-version: ["3.9", "3.10", "3.11", "3.12"]
49+
steps:
50+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
51+
- name: Set up Python ${{ matrix.python-version }}
52+
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
53+
with:
54+
python-version: ${{ matrix.python-version }}
55+
- name: Install dependencies
56+
run: |
57+
pip install -r requirements-dev.txt
58+
59+
- name: Checkout vllm-project/vllm repo
60+
uses: actions/checkout@v4
61+
with:
62+
repository: vllm-project/vllm
63+
path: vllm-empty
64+
65+
- name: Install vllm-project/vllm from source
66+
working-directory: vllm-empty
67+
run: |
68+
pip install -r requirements-build.txt --extra-index-url https://download.pytorch.org/whl/cpu
69+
VLLM_TARGET_DEVICE=empty pip install .
70+
71+
- name: Mypy
72+
run: |
73+
echo "::add-matcher::.github/workflows/matchers/mypy.json"
74+
tools/mypy.sh 1 ${{ matrix.python-version }}

‎.github/workflows/ruff.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
#
2+
# Adapted from vllm-project/vllm/blob/main/.github
3+
# Copyright 2023 The vLLM team.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
name: ruff
19+
20+
on:
21+
# Trigger the workflow on push or pull request,
22+
# but only for the main branch
23+
push:
24+
branches:
25+
- "main"
26+
paths:
27+
- "**/*.py"
28+
- requirements-lint.txt
29+
- .github/workflows/matchers/ruff.json
30+
- .github/workflows/ruff.yml
31+
pull_request:
32+
branches:
33+
- "main"
34+
35+
jobs:
36+
ruff:
37+
runs-on: ubuntu-latest
38+
strategy:
39+
matrix:
40+
python-version: ["3.12"]
41+
steps:
42+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
43+
- name: Set up Python ${{ matrix.python-version }}
44+
uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b # v5.3.0
45+
with:
46+
python-version: ${{ matrix.python-version }}
47+
- name: Install dependencies
48+
run: |
49+
python -m pip install --upgrade pip
50+
pip install -r requirements-lint.txt
51+
- name: Analysing the code with ruff
52+
run: |
53+
echo "::add-matcher::.github/workflows/matchers/ruff.json"
54+
ruff check --output-format github .
55+
- name: Run isort
56+
run: |
57+
isort . --check-only

‎.github/workflows/shellcheck.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
#
2+
# Adapted from vllm-project/vllm/blob/main/.github
3+
# Copyright 2023 The vLLM team.
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
18+
name: Lint shell scripts
19+
on:
20+
push:
21+
branches:
22+
- "main"
23+
paths:
24+
- '**/*.sh'
25+
- '.github/workflows/shellcheck.yml'
26+
pull_request:
27+
branches:
28+
- "main"
29+
paths:
30+
- '**/*.sh'
31+
- '.github/workflows/shellcheck.yml'
32+
33+
env:
34+
LC_ALL: en_US.UTF-8
35+
36+
defaults:
37+
run:
38+
shell: bash
39+
40+
permissions:
41+
contents: read
42+
43+
jobs:
44+
shellcheck:
45+
runs-on: ubuntu-latest
46+
steps:
47+
- name: "Checkout"
48+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
49+
with:
50+
fetch-depth: 0
51+
52+
- name: "Check shell scripts"
53+
run: |
54+
tools/shellcheck.sh

0 commit comments

Comments
 (0)