[Feature] Non-contiguous Support for FP8 Quantization

[yewentao256]

Purpose

Fix #19630

• using common vectorization utils
• support non-contiguous input (last stride == 1)
• empty instead of zeros

Test

Acc

lm_eval --model vllm --model_args "pretrained=nm-testing/DeepSeek-Coder-V2-Lite-Instruct-FP8,max_model_len=32768,enable_expert_parallel=True,enforce_eager=True" --trust_remote_code --tasks gsm8k --num_fewshot 5 --batch_size auto

# Now
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.7574|±  |0.0118|
|     |       |strict-match    |     5|exact_match|↑  |0.7354|±  |0.0122|

# main
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.7574|±  |0.0118|
|     |       |strict-match    |     5|exact_match|↑  |0.7354|±  |0.0122|

Perf

vllm bench throughput --model nm-testing/DeepSeek-Coder-V2-Lite-Instruct-FP8 --input-len 1000 --output-len 100 --trust_remote_code --enforce_eager

# Now
Throughput: 49.45 requests/s, 54306.45 total tokens/s, 4944.88 output tokens/s
# main
Throughput: 48.92 requests/s, 53730.95 total tokens/s, 4892.48 output tokens/s

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request introduces support for non-contiguous tensors in FP8 quantization kernels. However, the implementation of segmented_max_reduction_strided has critical correctness and performance issues that need to be addressed before merging.

[mgoin]

Nice work! Can you add at least one case to the unit test that expresses this type of non-contiguous input?

[mgoin]

@yewentao256 can you add a simple unit test for this before merge?

[yewentao256]

@yewentao256 can you add a simple unit test for this before merge?

Thanks for the comment! Done

(.wentao_env) wentao@gpu66:~/vllm-source/tests/quantization$ pytest test_fp8.py::test_scaled_fp8_quant
================================ test session starts ================================
platform linux -- Python 3.12.3, pytest-8.4.0, pluggy-1.6.0
rootdir: /home/wentao/vllm-source
configfile: pyproject.toml
plugins: asyncio-1.0.0, anyio-4.9.0
asyncio: mode=Mode.STRICT, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 2 items                                                                   

test_fp8.py ..                                                                [100%]

================================= 2 passed in 2.12s =================================