Skip to content

[Bugfix] Convert untraceable GroupShape to list for AMD impl #26535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 10, 2025
Merged

[Bugfix] Convert untraceable GroupShape to list for AMD impl #26535

merged 1 commit into from
Oct 10, 2025

Conversation

@Lucaskabela
Copy link
Contributor

@Lucaskabela Lucaskabela commented Oct 9, 2025 •
edited by github-actions bot
Loading

Purpose

#25696 converts GroupShape to list where possible for cutlass custom ops; however, this was not done for the aiter or triton impl, which causes the code to fail in dynamo tracing

Test Plan

Run DeepseekR1-0528 on AMD hardware with FP8 kernels

 FLASH_ATTENTION_TRITON_AMD_ENABLE=TRUE VLLM_USE_V1=1 VLLM_MLA_DISABLE=0 VLLM_FP8_PADDING=1 VLLM_USE_TRITON_FLASH_ATTN=1 VLLM_USE_ROCM_FP8_FLASH_ATTN=0 HSA_NO_SCRATCH_RECLAIM=1 
VLLM_USE_STANDALONE_COMPILE=1 with-proxy python examples/offline_inference/basic/generate.py --model=deepseek-ai/DeepSeek-R1-0528 --max-model-len=1024 -tp=8
Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@Lucaskabela Lucaskabela mentioned this pull request Oct 9, 2025
Merged
@mergify mergify bot added the rocm Related to AMD ROCm label Oct 9, 2025
@Lucaskabela
Copy link
Contributor Author

cc @ElizaWszola @zou3519 @ProExpertProg

@Lucaskabela Lucaskabela marked this pull request as ready for review October 9, 2025 21:14
@bradleyhd
Copy link
Contributor

can confirm this unblocks our internal AMD pipeline, thanks @Lucaskabela !

zou3519
zou3519 reviewed Oct 9, 2025
View reviewed changes
vllm/model_executor/layers/quantization/utils/fp8_utils.py
input_scale,
weight_scale,
self.weight_group_shape,
list(self.weight_group_shape),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to add a test somehow? I don't know how vLLM CI runs amd tests

@zou3519 zou3519 requested a review from ProExpertProg October 9, 2025 21:49
@zou3519 zou3519 added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 9, 2025
@zou3519
Copy link
Collaborator

zou3519 commented Oct 9, 2025

If someone has an idea for how to write a test for this please shout (I'm not very good with how AMD works in vLLM CI), otherwise this is pretty self contained and we verified that it fixes a compile regression

yewentao256
yewentao256 approved these changes Oct 9, 2025
View reviewed changes
Copy link
Member

@yewentao256 yewentao256 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the work!

ProExpertProg
ProExpertProg approved these changes Oct 9, 2025
View reviewed changes
Copy link
Collaborator

@ProExpertProg ProExpertProg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry we missed this initially, thanks for fixing! AMD testing is in pretty poor state cc @Alexei-V-Ivanov-AMD @gshtras

@zou3519 zou3519 enabled auto-merge (squash) October 10, 2025 11:14
@Lucaskabela @zou3519
Convert untraceable groupshape to list
7206439 
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
@zou3519 zou3519 force-pushed the lucaskabela/quant_dynamo_amd_fix branch from c9a82a6 to 7206439 Compare October 10, 2025 11:21
@zou3519 zou3519 merged commit 213b644 into vllm-project:main Oct 10, 2025
52 checks passed
Dhruvilbhatt pushed a commit to Dhruvilbhatt/vllm that referenced this pull request Oct 14, 2025
@Lucaskabela @Dhruvilbhatt
[Bugfix] Convert untraceable GroupShape to list for AMD impl (vllm-pr…
224c214 
…oject#26535)

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: Dhruvil Bhatt <bhattdbh@amazon.com>
bbartels pushed a commit to bbartels/vllm that referenced this pull request Oct 16, 2025
@Lucaskabela @bbartels
[Bugfix] Convert untraceable GroupShape to list for AMD impl (vllm-pr…
6a6131f 
…oject#26535)

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: bbartels <benjamin@bartels.dev>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@Lucaskabela @lywa1998
[Bugfix] Convert untraceable GroupShape to list for AMD impl (vllm-pr…
cf3120d 
…oject#26535)

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
@Lucaskabela @alhridoy
[Bugfix] Convert untraceable GroupShape to list for AMD impl (vllm-pr…
8889956 
…oject#26535)

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
@Lucaskabela @xuebwang-amd
[Bugfix] Convert untraceable GroupShape to list for AMD impl (vllm-pr…
39c9730 
…oject#26535)

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
0xrushi pushed a commit to 0xrushi/vllm that referenced this pull request Oct 26, 2025
@Lucaskabela @0xrushi
[Bugfix] Convert untraceable GroupShape to list for AMD impl (vllm-pr…
507fa9f 
…oject#26535)

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
0xrushi pushed a commit to 0xrushi/vllm that referenced this pull request Oct 26, 2025
@Lucaskabela @0xrushi
[Bugfix] Convert untraceable GroupShape to list for AMD impl (vllm-pr…
63491be 
…oject#26535)

Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zou3519 zou3519 zou3519 approved these changes

@ProExpertProg ProExpertProg ProExpertProg approved these changes

@yewentao256 yewentao256 yewentao256 approved these changes

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth is a code owner

Assignees

No one assigned

Labels

ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Lucaskabela @bradleyhd @zou3519 @ProExpertProg @yewentao256