Enable block-size > 1 by enable block table mapping

[ganyi1996ppo]

Purpose

Enable block-size > 1 by enable block table mapping

Test Plan

Test Result

# launch script

export VLLM_USE_V1=1
export SAFETENSORS_FAST_GPU=1
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_MOE=1
export VLLM_USE_TRITON_FLASH_ATTN=0
export NCCL_DEBUG=WARN
export VLLM_RPC_TIMEOUT=1800000
export VLLM_ROCM_USE_AITER_ASMMOE=1
export VLLM_ROCM_USE_AITER_MHA=0
export VLLM_ROCM_USE_TRITON_ROPE=1

# for profiling
# export VLLM_TORCH_PROFILER_DIR="deepseek_in3k_out1k"
# export VLLM_TORCH_PROFILER_WITH_STACK=1
# export VLLM_TORCH_PROFILER_RECORD_SHAPES=1

model_path="/mnt/raid0/zhangguopeng/deepseek-r1-FP8-Dynamic"
vllm serve $model_path \
  --tensor-parallel-size 8 \
  --max-num-batched-tokens 32768 \
  --trust-remote-code \
  --no-enable-prefix-caching \
  --disable-log-requests \
  --gpu_memory_utilization 0.9 \
  --block-size 128 \
  --compilation-config '{"cudagraph_mode": "FULL_AND_PIECEWISE"}'

# gsm8k test
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9507|±  |0.0060|
|     |       |strict-match    |     5|exact_match|↑  |0.9484|±  |0.0061|

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[tjtanaavllm]

My understanding is that AITER MLA decode is optimized for block-size 1.

Based on this understanding, I have three questions regarding to this PR:

1. Is this for DeepSeek V3.2?
2. Will there be performance gain when block-size > 1?
3. Are there new MLA decode kernels that support block-size > 1?

[tjtanaavllm]

The best for overall throughput would still be to set block-size=1.

Here's a comparison table of the benchmark results on DeepSeek-R1 PTPC FP8:

General Metrics

Metric	Before PR (Block-size 1)	After PR (Block-size 1)	After PR (Block-size 16)	Best Performance
General Performance
Successful requests	320	320	320	All equal
Benchmark duration (s)	359.51	354.92	360.31	Block-size 1
Total generated tokens	298,762	294,597	300,636	Block-size 16
Request throughput (req/s)	0.89	0.90	0.89	Block-size 1
Output token throughput (tok/s)	831.03	830.05	834.37	Block-size 16
Peak output token throughput (tok/s)	1,056.00	1,088.00	1,088.00	Block-size 1 & 16
Total token throughput (tok/s)	4,020.30	4,060.55	4,016.48	Block-size 1

Latency Metrics

Latency Metrics	Before PR	After PR (Block-size 1)	After PR (Block-size 16)	Best Performance
Mean TTFT (ms)	1,923.96	1,522.36	1,742.48	After PR (Block-size 1)
Median TTFT (ms)	1,686.80	1,411.06	1,655.13	After PR (Block-size 1)
P99 TTFT (ms)	5,553.48	5,530.39	5,531.85	After PR (Block-size 1)
Mean TPOT (ms)	57.56	53.10	58.82	After PR (Block-size 1)
Median TPOT (ms)	35.44	36.28	35.15	After PR (Block-size 16)
P99 TPOT (ms)	721.81	805.45	647.23	After PR (Block-size 16)
Mean ITL (ms)	35.03	35.79	34.86	Before PR
Median ITL (ms)	31.29	31.43	31.02	After PR (Block-size 16)
P99 ITL (ms)	209.77	211.47	210.17	Before PR

Workload

#!/bin/bash
PORT=8000
SEED=0
CONCURRENCY=32
NREQUESTS=$(($CONCURRENCY * 10))
ISL=3584
OSL=1024
vllm bench serve --backend vllm \
--model EmbeddedLLM/deepseek-r1-FP8-Dynamic \
--dataset-name random \
--num-prompts ${NREQUESTS} \
--random-input ${ISL} \
--random-output ${OSL} \
--seed ${SEED} \
--max-concurrency ${CONCURRENCY} --port ${PORT} \
| tee afterblocksizelgt1_blocksize16_EmbeddedLLM_deepseek-r1-FP8-Dynamic_v1_random_isl${ISL}_osl${OSL}_con${CONCURRENCY}.log

[tjtanaavllm]

LGTM. Amazing feature.