[Perf] Using __nv_fp8_e4m3 instead of c10::e4m3 for per_token_group_quant

[yewentao256]

Purpose

sgl-project/sglang#8499 shows we can get performance improvement through __nv_fp8_e4m3

This PR does that, doesn't change dynamic/static fp8 quant yet since we will update it together in #19630

Test

(vllm-user-6) vllm-user-6@centralia:~/vllm/benchmarks/kernels$ python benchmark_per_token_group_quant.py

Shape	col_major	ue8m0	Old Run (µs)	New Run (µs)	变化 (%)
(2, 8192)	0	0	13.731	13.530	-1.5% 🚀
	0	1	11.990	11.658	-2.8% 🚀
	1	0	13.254	13.277	+0.2% 🔺
	1	1	13.587	13.610	+0.2% 🔺
(4, 8192)	0	0	11.757	12.403	+5.5% 🔺
	0	1	12.022	12.352	+2.7% 🔺
	1	0	12.957	12.944	-0.1% 🚀
	1	1	13.114	12.653	-3.5% 🚀
(8, 8192)	0	0	11.866	11.494	-3.1% 🚀
	0	1	12.006	11.578	-3.6% 🚀
	1	0	13.466	12.813	-4.8% 🚀
	1	1	12.832	13.440	+4.7% 🔺
(16, 8192)	0	0	11.478	11.734	+2.2% 🔺
	0	1	11.386	11.366	-0.2% 🚀
	1	0	12.912	13.136	+1.7% 🔺
	1	1	12.979	12.710	-2.1% 🚀
(32, 8192)	0	0	11.504	11.827	+2.8% 🔺
	0	1	11.910	11.315	-5.0% 🚀
	1	0	13.408	12.966	-3.3% 🚀
	1	1	13.040	13.114	+0.6% 🔺
(64, 8192)	0	0	11.552	11.411	-1.2% 🚀
	0	1	11.347	11.526	+1.6% 🔺
	1	0	12.858	12.730	-1.0% 🚀
	1	1	13.094	13.120	+0.2% 🔺
(128, 8192)	0	0	11.709	11.360	-3.0% 🚀
	0	1	11.491	11.280	-1.8% 🚀
	1	0	12.749	12.714	-0.3% 🚀
	1	1	12.576	12.502	-0.6% 🚀
(256, 8192)	0	0	11.664	11.504	-1.4% 🚀
	0	1	11.846	11.290	-4.7% 🚀
	1	0	12.733	13.264	+4.2% 🔺
	1	1	12.781	12.445	-2.6% 🚀
(512, 8192)	0	0	11.706	11.469	-2.0% 🚀
	0	1	11.917	11.363	-4.7% 🚀
	1	0	13.053	12.864	-1.4% 🚀
	1	1	13.027	12.970	-0.4% 🚀
(1024, 8192)	0	0	17.936	15.677	-12.6% 🚀
	0	1	19.261	15.651	-18.7% 🚀
	1	0	19.802	17.664	-10.8% 🚀
	1	1	19.965	17.878	-10.5% 🚀
(2048, 8192)	0	0	31.690	25.853	-18.4% 🚀
	0	1	32.112	26.941	-16.1% 🚀
	1	0	32.086	28.230	-12.0% 🚀
	1	1	34.134	30.214	-11.5% 🚀
(4096, 8192)	0	0	60.486	50.432	-16.6% 🚀
	0	1	62.682	51.184	-18.3% 🚀
	1	0	62.806	56.611	-9.9% 🚀
	1	1	64.765	58.582	-9.6% 🚀
(8192, 8192)	0	0	114.080	93.894	-17.7% 🚀
	0	1	117.923	97.427	-17.4% 🚀
	1	0	120.314	108.122	-10.1% 🚀
	1	1	123.184	112.099	-9.0% 🚀
(16384, 8192)	0	0	221.286	181.498	-18.0% 🚀
	0	1	230.547	187.706	-18.6% 🚀
	1	0	234.848	210.230	-10.5% 🚀
	1	1	240.218	218.957	-8.8% 🚀

Acc Test

lm_eval --model vllm --model_args "pretrained=Qwen/Qwen3-30B-A3B-FP8,max_model_len=32768,enforce_eager=True" --trust_remote_code --tasks gsm8k --num_fewshot 5 --batch_size auto

main

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8234|±  |0.0105|
|     |       |strict-match    |     5|exact_match|↑  |0.8954|±  |0.0084|

Now:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8234|±  |0.0105|
|     |       |strict-match    |     5|exact_match|↑  |0.8954|±  |0.0084|

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request improves the performance of per-token-group FP8 quantization by replacing the C++-emulated c10::e4m3 type with the native CUDA __nv_fp8_e4m3 type. The benchmark results clearly show the benefits of this change. The review includes one high-severity suggestion to verify the removal of the header file doesn't introduce any regressions.

[mgoin]

Cool! Could you run the kernel benchmark on a more standard range of M, like 2^1 to 2^14, and an lm-eval check - I think Qwen3 FP8 block should be fine to trigger this

[yewentao256]

Cool! Could you run the kernel benchmark on a more standard range of M, like 2^1 to 2^14, and an lm-eval check - I think Qwen3 FP8 block should be fine to trigger this

Sure~ Updated in the above

[yewentao256]

@mgoin Updated using 8192 now