Skip to content

Conversation

@heheda12345
Copy link
Collaborator

@heheda12345 heheda12345 commented Oct 10, 2025

Purpose

After #24486 , deepseek 3.2 will throw this error:

 NotImplementedError: unknown kv cache spec UniformTypeKVCacheSpecs

This PR fix it.

FIX #26524

Test Plan

python3 examples/offline_inference/basic/generate.py --model deepseek-ai/DeepSeek-V3.2-Exp --gpu_memory_utilization 0.8 -tp 8

Test Result

--------------------------------------------------
Prompt: 'Hello, my name is'
Generated text: ' Christian Munoz and\nthis is my final project for my summer\n2020'
--------------------------------------------------
Prompt: 'The president of the United States is'
Generated text: ' the head of state and head of government of the United States, indirectly elected to'
--------------------------------------------------
Prompt: 'The capital of France is'
Generated text: ' Paris, and the capital of Spain is Madrid.\n\n**Question:** The capital of'
--------------------------------------------------
Prompt: 'The future of AI is'
Generated text: ' the future of work\n\nThe future of AI is the future of work\n\nThe'
--------------------------------------------------

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly adds support for UniformTypeKVCacheSpecs to address an issue with deepseek models. The approach of unpacking the spec and then handling it in the existing logic is sound. I've identified a minor improvement opportunity in the error reporting within gpu_model_runner.py to enhance debuggability for future unhandled spec types.

@heheda12345 heheda12345 added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 10, 2025
Copy link
Collaborator

@LucasWilkinson LucasWilkinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks for fixing!

@heheda12345 heheda12345 merged commit 6f0f570 into vllm-project:main Oct 10, 2025
53 of 54 checks passed
@heheda12345 heheda12345 deleted the fix_ds32_block_size branch October 10, 2025 08:40
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
…t#26559)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Dhruvilbhatt pushed a commit to Dhruvilbhatt/vllm that referenced this pull request Oct 14, 2025
…t#26559)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: Dhruvil Bhatt <bhattdbh@amazon.com>
bbartels pushed a commit to bbartels/vllm that referenced this pull request Oct 16, 2025
…t#26559)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: bbartels <benjamin@bartels.dev>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
…t#26559)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
0xrushi pushed a commit to 0xrushi/vllm that referenced this pull request Oct 26, 2025
…t#26559)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
0xrushi pushed a commit to 0xrushi/vllm that referenced this pull request Oct 26, 2025
…t#26559)

Signed-off-by: Chen Zhang <zhangch99@outlook.com>
Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek Related to DeepSeek models ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: prepare_kernel_block_sizes doesn't parse UniformTypeKVCacheSpecs

2 participants