Skip to content

Fp8 paged attention update #22222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 49 commits into from
Sep 15, 2025
Merged

Fp8 paged attention update #22222

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
419724f
Update attention.cu
xiao-llm Aug 5, 2025
ea0cef2
Update attention.cu
xiao-llm Aug 5, 2025
0165d16
Update attention.cu
xiao-llm Aug 5, 2025
9ba1300
Add unit test
Aug 6, 2025
360f62d
Remove dupicated test
Aug 6, 2025
dbfce63
Update CMakeLists.txt
xiao-llm Aug 6, 2025
b6bbaef
Update attention.cu
xiao-llm Aug 6, 2025
3849ff1
Fix some errors
amd-xiaoyu12 Aug 7, 2025
22b969c
Refactor with fp8::scaled_vec_convert
amd-xiaoyu12 Aug 7, 2025
b17b518
Clear comment
amd-xiaoyu12 Aug 7, 2025
3aac21e
Add mfma option
amd-xiaoyu12 Aug 12, 2025
b473aae
Fix mfma option bind error and passed test
amd-xiaoyu12 Aug 12, 2025
41cbcd2
Add template support for mfma type
amd-xiaoyu12 Aug 13, 2025
2066376
Update mfma option to string
amd-xiaoyu12 Aug 13, 2025
36d1f62
Update mfma option default value
amd-xiaoyu12 Aug 13, 2025
12cbc3b
cleanup code
amd-xiaoyu12 Aug 13, 2025
3601d88
Fix mfma type error
amd-xiaoyu12 Aug 13, 2025
f5a6cb5
Update attention.cu format
xiao-llm Aug 13, 2025
21e66d8
Update attention.cu for mfma type check
xiao-llm Aug 13, 2025
43c088a
Fix compile error for attention.cu
amd-xiaoyu12 Aug 13, 2025
d44d1d9
Update benchmark_paged_attention.py
xiao-llm Aug 13, 2025
8a62a4e
Clean debug code
amd-xiaoyu12 Aug 26, 2025
a0b7735
Support FP8 mfma env setting
amd-xiaoyu12 Aug 26, 2025
ed32461
Support VLLM_USE_FP8_MFMA in envs
amd-xiaoyu12 Aug 26, 2025
e425722
Refactor code for mfma type control
amd-xiaoyu12 Aug 26, 2025
377d43b
Fix attention ops interface error
amd-xiaoyu12 Aug 27, 2025
e01bd22
Revert "[Bugfix][V1][P/D]Fix the uneven polling issue in the toy prox…
amd-xiaoyu12 Aug 27, 2025
73867f9
Update _custom_ops.py
xiao-llm Aug 27, 2025
9b7d696
Update vllm/envs.py
xiao-llm Aug 28, 2025
096b4ce
Update envs.py
xiao-llm Aug 28, 2025
dc106f3
clean up space
amd-xiaoyu12 Aug 28, 2025
38514d3
Cleanup
amd-xiaoyu12 Aug 28, 2025
93adf9b
Cleaup rebase conflicts
amd-xiaoyu12 Aug 28, 2025
964a10d
Update disagg_proxy_p2p_nccl_xpyd.py
xiao-llm Aug 28, 2025
53bcf85
Update disagg_proxy_p2p_nccl_xpyd.py
xiao-llm Aug 28, 2025
43b7f8b
Update disagg_proxy_p2p_nccl_xpyd.py
xiao-llm Aug 28, 2025
2bf823c
Correct env flag name
amd-xiaoyu12 Aug 28, 2025
8aa63ef
Remvoe unused mfma option due to rebase
amd-xiaoyu12 Aug 29, 2025
1b8bb96
Update for compile flags
amd-xiaoyu12 Sep 2, 2025
f7c1024
refactoring fp8 mfma compile flag
xiao-llm Sep 9, 2025
3f306d8
Correct fp8 mfma compile flag after refactoring
xiao-llm Sep 9, 2025
bdb3a4d
Correct fp8 mfma compile flag location
xiao-llm Sep 9, 2025
e3d4ea2
Update envs fp8 mfma check
xiao-llm Sep 9, 2025
6cd5b28
Update envs.py
xiao-llm Sep 9, 2025
e7b1aa6
Update _custom_ops.py
xiao-llm Sep 10, 2025
adac5b4
Reformat attention.cu
xiao-llm Sep 10, 2025
478b3d8
Merge branch 'main' into fp8_paged_attention_update
xiao-llm Sep 10, 2025
c945bd0
Resolve envs.py conflit
xiao-llm Sep 14, 2025
129a59f
Merge branch 'main' into fp8_paged_attention_update
xiao-llm Sep 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading