Skip to content

[aclgraph] implentment NPUPiecewiseBackend to enable aclgraph #836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
May 29, 2025
Merged

[aclgraph] implentment NPUPiecewiseBackend to enable aclgraph #836

merged 3 commits into from
May 29, 2025

Conversation

@MengqingCao
Copy link
Collaborator

@MengqingCao MengqingCao commented May 13, 2025 •
edited
Loading

What this PR does / why we need it?

  1. Implentment NPUPiecewiseBackend to enable aclgraph
  2. Eable aclgraph by default in V1, but raise error when running deepseek and raise warning when running models except for qwen

How was this patch tested?

CI pass with the new ut

@MengqingCao MengqingCao mentioned this pull request May 13, 2025
Merged
@MengqingCao MengqingCao marked this pull request as draft May 13, 2025 13:00
@yiz-liu
Copy link
Collaborator

yiz-liu commented May 23, 2025

I believe this is acceptable overall; however, we need to standardize our use of either npugraph or aclgraph throughout the codebase. At present, I use aclgraph consistently and recommend that we continue doing so moving forward.

Additionally, some comments still reference “cuda,” which should be updated to “ascend” or “npu.”

@MengqingCao

@MengqingCao
Copy link
Collaborator Author

I believe this is acceptable overall; however, we need to standardize our use of either npugraph or aclgraph throughout the codebase. At present, I use aclgraph consistently and recommend that we continue doing so moving forward.

Additionally, some comments still reference “cuda,” which should be updated to “ascend” or “npu.”

@MengqingCao

Good catch, the latest commit has changed to use aclgraph, and the comments have been fixed

@MengqingCao MengqingCao marked this pull request as ready for review May 23, 2025 08:22
@MengqingCao MengqingCao force-pushed the piecewise branch 2 times, most recently from 23e1738 to c2c5403 Compare May 27, 2025 01:47
@MengqingCao
Copy link
Collaborator Author

@yiz-liu @wangxiyuan could you take a look at the latest code?

Currently, aclgraph is enabled in V1 by default, and will raise error if running deepseek and throw warning if running models except for qwen.

wangxiyuan
wangxiyuan reviewed May 27, 2025
View reviewed changes
Copy link
Collaborator

@wangxiyuan wangxiyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please rebase after #952 merged.

@MengqingCao
[aclgraph] implentment NPUPiecewiseBackend to enable aclgraph
163124b 
Signed-off-by: MengqingCao <cmq0113@163.com>
@MengqingCao
skip aclgraph ut in 0.8.5
e273e03 
Signed-off-by: MengqingCao <cmq0113@163.com>
@MengqingCao
Copy link
Collaborator Author

@wangxiyuan thanks for your review and all the comments are addressed now. Could this be merged now?

wangxiyuan
wangxiyuan reviewed May 28, 2025
View reviewed changes
@MengqingCao
fix review
3c7e526 
Signed-off-by: MengqingCao <cmq0113@163.com>
@wangxiyuan wangxiyuan merged commit a93bed4 into vllm-project:main May 29, 2025
22 checks passed
raindaywhu added a commit to raindaywhu/vllm-ascend that referenced this pull request May 30, 2025
@raindaywhu
Merge branch 'main' of https://github.com/raindaywhu/vllm-ascend into…
fd7b81e 
… main

* 'main' of https://github.com/raindaywhu/vllm-ascend:
  [aclgraph] implentment NPUPiecewiseBackend to enable aclgraph (vllm-project#836)
  [Bugfix][V1] Fix deepseek with v1 (vllm-project#958)
  [Perf] Refactor tensor disposal logic to reduce memory usage (vllm-project#966)
David9857 pushed a commit to David9857/vllm-ascend that referenced this pull request Jun 3, 2025
@MengqingCao @David9857
[aclgraph] implentment NPUPiecewiseBackend to enable aclgraph (vllm-p…
f7cc6fe 
…roject#836)

### What this PR does / why we need it?
1. Implentment `NPUPiecewiseBackend` to enable aclgraph
2. Eable aclgraph by default in V1, but raise error when running
deepseek and raise warning when running models except for qwen

### How was this patch tested?
CI pass with the new ut

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
@MengqingCao MengqingCao deleted the piecewise branch June 4, 2025 12:35
@MengqingCao MengqingCao mentioned this pull request Jun 12, 2025
Open
83 tasks
@wangxiyuan wangxiyuan mentioned this pull request Oct 13, 2025
Merged
wangxiyuan added a commit that referenced this pull request Oct 14, 2025
@wangxiyuan
[Community] Nominate new maintainers: @yiz-liu @paulyu12 @weijinqian0 @…
49b8502 
…nalinaly (#3406)

I'd like to nominate 4 new maintainers for vllm-ascend: 

----

Yizhou Liu [@yiz-liu](https://github.com/yiz-liu)
----

**Review Quality‌**: He has completed [40+
reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+commenter%3Ayiz-liu)
and provided solutions or guides for [10+
issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20commenter%3Ayiz-liu),
which includes many quality review like
[#issue-3428408401](#3002 (comment)),
[#discussion_r2224572309](#1803 (comment)),
[#issuecomment-2982470226](#1261 (comment)),
[#issuecomment-2903621197](#836 (comment)),
[#issuecomment-2857678691](#778 (comment)).

**Sustained and High-Quality Contributions:** He has contributed more
than [30+
commits](https://github.com/vllm-project/vllm-ascend/commits?author=yiz-liu)
since Mar.2025, especially, aclgraph, DP, and EP related contributions
are the main reason why I nominated him. As the owner of aclgraph
support, he continuously improves aclgraph stability and performance as
well as fixes key bugs. he laid the groundwork for EP-related
functionality and delivered multiple foundational improvements

**Community involvement:** He has a very good habit of logging
issues：#1649 and is
also very active and involved in [many
issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aopen%20commenter%3Ayiz-liu%20-author%3Ayiz-liu)
to help users resolve issues.

----

Peng Yu  [@paulyu12](https://github.com/paulyu12)
---
The main reasons for his nomination are his expertise and key
contributions to the LORA and sustained and major contributions (initial
support/doc/bugfix) around Lora.

**Sustained and Major Contributions:** @paulyu12 starts his contribution
with [Lora and Mulit-Lora
support](697908f)
since Apr 2025, he contributed about [10+ commits and
bugfixes](697908f)
on vllm-ascend.
**Review Quality‌ and Community Involvement‌:** He also helped more than
10+ users address [Lora related
issues](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+commenter%3Apaulyu12+-author%3Apaulyu12+is%3Aclosed).

I believe his addition will further improve vLLM Ascend Lora support.

----

Jinqian Wei [@weijinqian0](https://github.com/weijinqian0)
---
The main reasons for his nomination are his key contributions to the RL
scene and the high quality of his code reviews.

**Review Quality‌:** He has completed [60+
reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+commenter%3Aweijinqian0+is%3Aopen+-author%3Aweijinqian0)
since June. 2025, include
[#comment-3284055430](#2791 (comment)),
[discussion_r2332166704](#2817 (comment)),
[discussion_r2343289692](#2846 (comment))
high quality review.

**Sustained and Quality Contributions:** He has Deep understanding of
‌vLLM‌ and ‌vLLM Ascend‌ codebases and solid contributions in RL scene
(about [10+ PR
merged](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3Aweijinqian0+is%3Amerged+)
and 10+ PRs merged as co-author.

- Code Refactor: As a co-author, he participated in the refactoring of
the MOE module #2150
#2706
#2867
- Performance Enhancement for RL: Participated as a co-author in the
design and development of the solution, contributing to the planning of
core capabilities. #1547
#2120 and so on.

So I think he's a great addition to the vLLM Ascend Maintainer team.

----

Chuanyu Qin  [@nalinaly](https://github.com/nalinaly)
---
The main reason I nominated Qinchuanyu is because he is the initial
designer of aclgraph and torch-npu, two key components of vllm-ascend.
Considering aclgraph will eventually become the main path for
vllm-ascend's graph model, I propose to nominate him.

**Sustained and Major Contributions:** In fact, chuanyu actively helped
the users/developers of vllm-ascend since Mar 2025
([vllm-discuss#162](https://discuss.vllm.ai/t/can-ascend-officially-draft-a-documentation-on-the-vllm-ascend-adaptation-for-graph-mode/162/5)),
and also helped early users of vllm-ascend understand aclgraph. He
provided lots of help in the process of integrating aclgraph with
vllm-ascend.

**Community Involvement‌:** As speaker, he also presents help users
understand aclgraph and torch_npu [《The design philosophy of torch_npu
and the high performance principle of
aclGraph》](https://github.com/PyTorch-China/pytorch-meetup/blob/main/beijing-2025/%E3%80%905%E3%80%91torch_npu%20%E7%9A%84%E8%AE%BE%E8%AE%A1%E5%93%B2%E5%AD%A6%E4%B8%8E%20aclGraph%20%E9%AB%98%E6%80%A7%E8%83%BD%E5%8E%9F%E7%90%86-%E7%A7%A6%E4%BC%A0%E7%91%9C-0920.pdf)

----

They have activate contribution to vllm-ascend or have rich experience
for ascend AI.

Welcome!
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
chopper0126 pushed a commit to chopper0126/vllm-ascend that referenced this pull request Oct 16, 2025
@MengqingCao
[aclgraph] implentment NPUPiecewiseBackend to enable aclgraph (vllm-p…
4d7e95d 
…roject#836)

### What this PR does / why we need it?
1. Implentment `NPUPiecewiseBackend` to enable aclgraph
2. Eable aclgraph by default in V1, but raise error when running
deepseek and raise warning when running models except for qwen

### How was this patch tested?
CI pass with the new ut

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
Angazenn pushed a commit to Angazenn/vllm-ascend that referenced this pull request Oct 21, 2025
@MengqingCao @Angazenn
[aclgraph] implentment NPUPiecewiseBackend to enable aclgraph (vllm-p…
0e0db5b 
…roject#836)

### What this PR does / why we need it?
1. Implentment `NPUPiecewiseBackend` to enable aclgraph
2. Eable aclgraph by default in V1, but raise error when running
deepseek and raise warning when running models except for qwen

### How was this patch tested?
CI pass with the new ut

---------

Signed-off-by: MengqingCao <cmq0113@163.com>
Angazenn pushed a commit to Angazenn/vllm-ascend that referenced this pull request Oct 21, 2025
@wangxiyuan @Angazenn
[Community] Nominate new maintainers: @yiz-liu @paulyu12 @weijinqian0 @…
a874821 
…nalinaly (vllm-project#3406)

I'd like to nominate 4 new maintainers for vllm-ascend: 

----

Yizhou Liu [@yiz-liu](https://github.com/yiz-liu)
----

**Review Quality‌**: He has completed [40+
reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+commenter%3Ayiz-liu)
and provided solutions or guides for [10+
issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20commenter%3Ayiz-liu),
which includes many quality review like
[#issue-3428408401](vllm-project#3002 (comment)),
[#discussion_r2224572309](vllm-project#1803 (comment)),
[#issuecomment-2982470226](vllm-project#1261 (comment)),
[#issuecomment-2903621197](vllm-project#836 (comment)),
[#issuecomment-2857678691](vllm-project#778 (comment)).

**Sustained and High-Quality Contributions:** He has contributed more
than [30+
commits](https://github.com/vllm-project/vllm-ascend/commits?author=yiz-liu)
since Mar.2025, especially, aclgraph, DP, and EP related contributions
are the main reason why I nominated him. As the owner of aclgraph
support, he continuously improves aclgraph stability and performance as
well as fixes key bugs. he laid the groundwork for EP-related
functionality and delivered multiple foundational improvements

**Community involvement:** He has a very good habit of logging
issues：vllm-project#1649 and is
also very active and involved in [many
issues](https://github.com/vllm-project/vllm-ascend/issues?q=is%3Aissue%20state%3Aopen%20commenter%3Ayiz-liu%20-author%3Ayiz-liu)
to help users resolve issues.

----

Peng Yu  [@paulyu12](https://github.com/paulyu12)
---
The main reasons for his nomination are his expertise and key
contributions to the LORA and sustained and major contributions (initial
support/doc/bugfix) around Lora.

**Sustained and Major Contributions:** @paulyu12 starts his contribution
with [Lora and Mulit-Lora
support](vllm-project@697908f)
since Apr 2025, he contributed about [10+ commits and
bugfixes](vllm-project@697908f)
on vllm-ascend.
**Review Quality‌ and Community Involvement‌:** He also helped more than
10+ users address [Lora related
issues](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+commenter%3Apaulyu12+-author%3Apaulyu12+is%3Aclosed).

I believe his addition will further improve vLLM Ascend Lora support.

----

Jinqian Wei [@weijinqian0](https://github.com/weijinqian0)
---
The main reasons for his nomination are his key contributions to the RL
scene and the high quality of his code reviews.

**Review Quality‌:** He has completed [60+
reviews](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+commenter%3Aweijinqian0+is%3Aopen+-author%3Aweijinqian0)
since June. 2025, include
[#comment-3284055430](vllm-project#2791 (comment)),
[discussion_r2332166704](vllm-project#2817 (comment)),
[discussion_r2343289692](vllm-project#2846 (comment))
high quality review.

**Sustained and Quality Contributions:** He has Deep understanding of
‌vLLM‌ and ‌vLLM Ascend‌ codebases and solid contributions in RL scene
(about [10+ PR
merged](https://github.com/vllm-project/vllm-ascend/pulls?q=is%3Apr+author%3Aweijinqian0+is%3Amerged+)
and 10+ PRs merged as co-author.

- Code Refactor: As a co-author, he participated in the refactoring of
the MOE module vllm-project#2150
vllm-project#2706
vllm-project#2867
- Performance Enhancement for RL: Participated as a co-author in the
design and development of the solution, contributing to the planning of
core capabilities. vllm-project#1547
vllm-project#2120 and so on.

So I think he's a great addition to the vLLM Ascend Maintainer team.

----

Chuanyu Qin  [@nalinaly](https://github.com/nalinaly)
---
The main reason I nominated Qinchuanyu is because he is the initial
designer of aclgraph and torch-npu, two key components of vllm-ascend.
Considering aclgraph will eventually become the main path for
vllm-ascend's graph model, I propose to nominate him.

**Sustained and Major Contributions:** In fact, chuanyu actively helped
the users/developers of vllm-ascend since Mar 2025
([vllm-discuss#162](https://discuss.vllm.ai/t/can-ascend-officially-draft-a-documentation-on-the-vllm-ascend-adaptation-for-graph-mode/162/5)),
and also helped early users of vllm-ascend understand aclgraph. He
provided lots of help in the process of integrating aclgraph with
vllm-ascend.

**Community Involvement‌:** As speaker, he also presents help users
understand aclgraph and torch_npu [《The design philosophy of torch_npu
and the high performance principle of
aclGraph》](https://github.com/PyTorch-China/pytorch-meetup/blob/main/beijing-2025/%E3%80%905%E3%80%91torch_npu%20%E7%9A%84%E8%AE%BE%E8%AE%A1%E5%93%B2%E5%AD%A6%E4%B8%8E%20aclGraph%20%E9%AB%98%E6%80%A7%E8%83%BD%E5%8E%9F%E7%90%86-%E7%A7%A6%E4%BC%A0%E7%91%9C-0920.pdf)

----

They have activate contribution to vllm-ascend or have rich experience
for ascend AI.

Welcome!
- vLLM version: v0.11.0rc3
- vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wangxiyuan wangxiyuan wangxiyuan approved these changes

Assignees

No one assigned

Labels

module:core module:ops module:tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@MengqingCao @yiz-liu @wangxiyuan