[Feature]: Optimize per_block_cast_to_fp8 #24607
Description
🚀 The feature, motivation and pitch
vllm/vllm/utils/deep_gemm.py
Currently the per_block_cast_to_fp8 is written by pytorch script, which would be slow
We can optimize it using triton kernel or cuda kernel
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.