Skip to content

feat: support cuda graph for batched multi-query(prefill/append) attention#277

Merged
yzh119 merged 7 commits intomainfrom prefill-cuda-graph-newJun 2, 2024

Commits

Commits on Jun 2, 2024