[TRITON] Support gfx1201 for triton gemm_a8w8_blockscale#1829
Open
big-yellow-duck wants to merge 29 commits intoROCm:mainfrom
Open
[TRITON] Support gfx1201 for triton gemm_a8w8_blockscale#1829big-yellow-duck wants to merge 29 commits intoROCm:mainfrom
big-yellow-duck wants to merge 29 commits intoROCm:mainfrom
Conversation
Co-authored-by: NAME Amir Balwel amoooori04@gmail.com
Co-authoured-by: Amir Balwel amoooori04@gmail.com
…aiter into support_gfx1201_min
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
…aiter into support_gfx1201_min
Co-authored-by: Jeff Aw <jeffaw99@hotmail.com> Signed-off-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: Jeff Aw <jeffaw99@hotmail.com> Signed-off-by: Amir Balwel <amoooori04@gmail.com>
Co-authored-by: Amir Balwel <amoooori04@gmail.com>
…aiter into support_gfx1201_min
big-yellow-duck
changed the title
azaidy
changed the title
Author
Using aiter gemm_w8a8 kernels in vllm shows performance uplift for gfx1201 when running Qwen3-0.6B-FP8 at higher input and ouput tokens, |
Signed-off-by: Amir Balwel <amoooori04@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This adds preliminary support for gfx1201 to use
gemm_a8w8_blockscalefrom triton which is used in Qwen/Qwen3-0.6B-FP8Moving forward, more triton kernels can be tuned to optimize the performance of gfx1201.
Technical Details
num_warpsandwaves_per_euto find the optimal execution time for a set of operations.Test Plan
test the tuned configs using
aiter/op_tests/triton_tests/gemm/basic/test_gemm_a8w8_blockscale.pyTest Result
126 tests have passed
2 skipped, (where N or K don't meet preshuffle kernel constraints: N must be multiple of 16, K must be multiple of 32)
Submission Checklist