[Model][Qwen3VL] Compute cu_seqlens on CPU to remove

[lgeiger]

Purpose

The current code does 5 cudaStreamSynchronize( 3 syncs due to CPU-GPU weight copy and 2 syncs inside torch.repeat_interleave) which isn't idea.
This PR removes the blocking weight copy and moves the cu_seqlens to the CPU which is faster and doesn't require any CPU-GPU syncs.

Before:

After:

Test Plan

On a single H100

vllm serve Qwen/Qwen3-VL-30B-A3B-Instruct-FP8 --limit-mm-per-prompt.video 0 --no-enable-prefix-caching

vllm bench serve --backend openai-chat --model Qwen/Qwen3-VL-30B-A3B-Instruct-FP8 --endpoint /v1/chat/completions --dataset-name hf --dataset-path lmarena-ai/VisionArena-Chat --hf-split train --num-prompts 1000

Test Result

Before:

============ Serving Benchmark Result ============
Successful requests:                     984
Benchmark duration (s):                  31.49
Total input tokens:                      93876
Total generated tokens:                  118504
Request throughput (req/s):              31.25
Output token throughput (tok/s):         3763.75
Peak output token throughput (tok/s):    10155.00
Peak concurrent requests:                984.00
Total Token throughput (tok/s):          6745.30
---------------Time to First Token----------------
Mean TTFT (ms):                          12682.91
Median TTFT (ms):                        11258.48
P99 TTFT (ms):                           26015.58
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          111.32
Median TPOT (ms):                        121.03
P99 TPOT (ms):                           257.56
---------------Inter-token Latency----------------
Mean ITL (ms):                           105.87
Median ITL (ms):                         63.28
P99 ITL (ms):                            447.10
==================================================

After:

============ Serving Benchmark Result ============
Successful requests:                     984
Benchmark duration (s):                  31.47
Total input tokens:                      93720
Total generated tokens:                  118619
Request throughput (req/s):              31.27
Output token throughput (tok/s):         3769.80
Peak output token throughput (tok/s):    10951.00
Peak concurrent requests:                984.00
Total Token throughput (tok/s):          6748.29
---------------Time to First Token----------------
Mean TTFT (ms):                          12383.31
Median TTFT (ms):                        11419.97
P99 TTFT (ms):                           26081.61
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          112.25
Median TPOT (ms):                        121.49
P99 TPOT (ms):                           248.47
---------------Inter-token Latency----------------
Mean ITL (ms):                           107.45
Median ITL (ms):                         61.96
P99 ITL (ms):                            486.04
==================================================

This reduces Mean TTFT by 1 - 2 %.

[gemini-code-assist]

Code Review

This pull request aims to improve performance by making tensor copies non-blocking and moving cu_seqlens computation to the CPU. The changes are generally well-implemented and align with the goal of reducing synchronization overhead. However, there is a critical issue where a tensor is moved to the GPU without reassigning the result to the variable, which will lead to either a runtime error or negate the intended performance benefit. I've left a specific comment with a suggested fix.