Skip to content

add custom ascendc kernel vocabparallelembedding #796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Jun 12, 2025
Merged

add custom ascendc kernel vocabparallelembedding #796

merged 17 commits into from
Jun 12, 2025

Conversation

@ttanzhiqiang
Copy link
Contributor

This PR add custom ascendc kernel vocabparallelembedding support in vllm-ascend, related CMakeLists and setuptools is also added in this PR.

pytest -s benchmarks/ops/ben_vocabparallelembedding.py
截屏2025-05-08 18 12 23

pytest -s tests/ops/test_vocabparallelembedding.py
截屏2025-05-08 18 13 31

@ttanzhiqiang
add vocabparallelembedding
cb880a7 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
add vocabparallelembedding
0c18a45 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
add debug message
2cc2993 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Initialize temporary buffers and benchmark
46aba85 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
update CMakeLists.txt get_masked_input_and_mask_kernel.cpp
3b14493 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
update benchmarks assert_close
893e0fe 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
ttanzhiqiang and others added 5 commits May 8, 2025 19:04
@ttanzhiqiang
bash format.sh
2da1233 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
bash format.sh
75da13c 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Removed f
bf0f319 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
isort file
9796700 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Merge branch 'vllm-project:main' into Custom_op
0f0a349
@ttanzhiqiang
Copy link
Contributor Author

@wangxiyuan @ganyi1996ppo

@wangxiyuan
Copy link
Collaborator

@ganyi1996ppo can you take a deep review for the custom ops change?

@ganyi1996ppo
Copy link
Collaborator

sure, I'll take a look

@ttanzhiqiang
Copy link
Contributor Author

@wangxiyuan @ganyi1996ppo

ttanzhiqiang and others added 4 commits May 29, 2025 11:55
@ttanzhiqiang
Resolving conflicts
9f864e3 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Resolving conflicts
c4d3e2b 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Resolving conflicts
63a2e89 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Merge branch 'main' into Custom_op
15c0f6a
@ttanzhiqiang
Copy link
Contributor Author

@wangxiyuan @ganyi1996ppo

ganyi1996ppo
ganyi1996ppo reviewed May 30, 2025
View reviewed changes
ganyi1996ppo
ganyi1996ppo reviewed May 30, 2025
View reviewed changes
@ttanzhiqiang
update Code Notes
f880ee5 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Copy link
Contributor Author

update @ganyi1996ppo

@wangxiyuan wangxiyuan mentioned this pull request Jun 4, 2025
Closed
76 tasks
@shiyuan680
Copy link
Contributor

i met such problem
20250606-170747(WeLinkPC)
npu0
input_shape torch.Size([410])
org_start: 0
org_end: 76032
padding: 0
added_start:152064
added_end:152064
npu1
input_shape torch.Size([410])
org_start: 76032
org_end: 152064
padding: 0
added_start:: 152064
added_end:152064

@ttanzhiqiang
Copy link
Contributor Author

i met such problem 20250606-170747(WeLinkPC) npu0 input_shape torch.Size([410]) org_start: 0 org_end: 76032 padding: 0 added_start:152064 added_end:152064 npu1 input_shape torch.Size([410]) org_start: 76032 org_end: 152064 padding: 0 added_start:: 152064 added_end:152064

ok i will try it

@ttanzhiqiang
update TORCH_CHECK
9201f45 
Signed-off-by: ttanzhiqiang <389825161@qq.com>
@ttanzhiqiang
Copy link
Contributor Author

i met such problem 20250606-170747(WeLinkPC) npu0 input_shape torch.Size([410]) org_start: 0 org_end: 76032 padding: 0 added_start:152064 added_end:152064 npu1 input_shape torch.Size([410]) org_start: 76032 org_end: 152064 padding: 0 added_start:: 152064 added_end:152064

@shiyuan680
update TORCH_CHECK
TORCH_CHECK(input.dim() >= 1, "input must have at least 1 dimension");
TORCH_CHECK(org_vocab_start_index >= 0, "org_vocab_start_index must be non-negative");
TORCH_CHECK(org_vocab_end_index >= org_vocab_start_index, "org_vocab_end_index must be greater than org_vocab_start_index");
TORCH_CHECK(num_org_vocab_padding >= 0, "num_org_vocab_padding must be non-negative");
TORCH_CHECK(added_vocab_start_index >= org_vocab_end_index, "added_vocab_start_index must be greater than org_vocab_end_index");
TORCH_CHECK(added_vocab_end_index >= added_vocab_start_index, "added_vocab_end_index must be greater than added_vocab_start_index");

Modify tests/ops/test_vocabparallelembedding.py and benchmarks/ops/ben_vocabparallelembedding.py
test_case = {
"org_start": 0,
"org_end": 76032,
"padding": 0,
"added_start": 152064,
"added_end": 152064,
}

@ApsarasX ApsarasX added the ready read for review label Jun 9, 2025
@wangxiyuan
Copy link
Collaborator

No feedback for long time. I'll merge this first.

@wangxiyuan wangxiyuan merged commit 2498d29 into vllm-project:main Jun 12, 2025
24 checks passed
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 17, 2025
@ttanzhiqiang
add custom ascendc kernel vocabparallelembedding (vllm-project#796)
3966fbf 
This PR add custom ascendc kernel vocabparallelembedding support in
vllm-ascend, related CMakeLists and setuptools is also added in this PR.

pytest -s benchmarks/ops/ben_vocabparallelembedding.py
pytest -s tests/ops/test_vocabparallelembedding.py

---------

Signed-off-by: ttanzhiqiang <389825161@qq.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 17, 2025
@ttanzhiqiang
add custom ascendc kernel vocabparallelembedding (vllm-project#796)
6794647 
This PR add custom ascendc kernel vocabparallelembedding support in
vllm-ascend, related CMakeLists and setuptools is also added in this PR.

pytest -s benchmarks/ops/ben_vocabparallelembedding.py
pytest -s tests/ops/test_vocabparallelembedding.py

---------

Signed-off-by: ttanzhiqiang <389825161@qq.com>
Signed-off-by: wangxiaoxin (A) <wangxiaoxin7@huawei.com>
momo609 pushed a commit to momo609/vllm-ascend that referenced this pull request Jun 17, 2025
@ttanzhiqiang
add custom ascendc kernel vocabparallelembedding (vllm-project#796)
fcd5ad8 
This PR add custom ascendc kernel vocabparallelembedding support in
vllm-ascend, related CMakeLists and setuptools is also added in this PR.

pytest -s benchmarks/ops/ben_vocabparallelembedding.py
pytest -s tests/ops/test_vocabparallelembedding.py

---------

Signed-off-by: ttanzhiqiang <389825161@qq.com>
Signed-off-by: wangxiaoxin (A) <wangxiaoxin7@huawei.com>
shiyuan680 pushed a commit to raindaywhu/vllm-ascend that referenced this pull request Jul 7, 2025
@ttanzhiqiang
add custom ascendc kernel vocabparallelembedding (vllm-project#796)
91889d3 
This PR add custom ascendc kernel vocabparallelembedding support in
vllm-ascend, related CMakeLists and setuptools is also added in this PR.

pytest -s benchmarks/ops/ben_vocabparallelembedding.py
pytest -s tests/ops/test_vocabparallelembedding.py

---------

Signed-off-by: ttanzhiqiang <389825161@qq.com>
chopper0126 pushed a commit to chopper0126/vllm-ascend that referenced this pull request Oct 16, 2025
@ttanzhiqiang
add custom ascendc kernel vocabparallelembedding (vllm-project#796)
ce80b3e 
This PR add custom ascendc kernel vocabparallelembedding support in
vllm-ascend, related CMakeLists and setuptools is also added in this PR.

pytest -s benchmarks/ops/ben_vocabparallelembedding.py
pytest -s tests/ops/test_vocabparallelembedding.py

---------

Signed-off-by: ttanzhiqiang <389825161@qq.com>
Angazenn pushed a commit to Angazenn/vllm-ascend that referenced this pull request Oct 21, 2025
@ttanzhiqiang @Angazenn
add custom ascendc kernel vocabparallelembedding (vllm-project#796)
3319860 
This PR add custom ascendc kernel vocabparallelembedding support in
vllm-ascend, related CMakeLists and setuptools is also added in this PR.

pytest -s benchmarks/ops/ben_vocabparallelembedding.py
pytest -s tests/ops/test_vocabparallelembedding.py

---------

Signed-off-by: ttanzhiqiang <389825161@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ganyi1996ppo ganyi1996ppo ganyi1996ppo left review comments

@wangxiyuan wangxiyuan wangxiyuan approved these changes

Assignees

No one assigned

Labels

module:tests ready read for review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@ttanzhiqiang @wangxiyuan @ganyi1996ppo @shiyuan680 @ApsarasX