[Bug]: Deepseek reasoning and guided_json no longer works

[supersteves]

Your current environment

The output of `python collect_env.py`

N/A. 

I'm using docker latest:

docker pull vllm/vllm-openai:latest
latest: Pulling from vllm/vllm-openai
...
Digest: sha256:4d8d397a62c36237293a4d5e2acbf911b91b0a8552825bda69f581c5811af9ec
Status: Downloaded newer image for vllm/vllm-openai:latest

And I'm running as follows:

docker run --runtime nvidia --gpus all     -v ~/.cache/huggingface:/root/.cache/huggingface     --env "HUGGING_FACE_HUB_TOKEN=xxx"  --env VLLM_LOGGING_LEVEL=DEBUG   -p 8000:8000     --ipc=host     vllm/vllm-openai:latest     --model deepseek-ai/DeepSeek-R1-Distill-Llama-70B --enforce-eager --tensor-parallel-size=4 --enable-reasoning --reasoning-parser deepseek_r1 

🐛 Describe the bug

I'm using the exact same example of Reasoning with Structured Outputs from the vLLM docs:
https://docs.vllm.ai/en/v0.8.1/features/reasoning_outputs.html#structured-output

I expect to see both reasoning and content. Something like this:

reasoning_content:  "Hmm, let me think for a bit... Wait, let me think a bit more..."
content:  {"name": "Ethan", "age": 28}

Instead I get:

reasoning_content:  {"name": "Ethan", "age": 28}
content:  None

(I also tried guided_grammar, although it's not documented as supported with reasoning in the top of the same page. With that case, it worked, but was unbearably slow, 100% of one CPU, and didn't utilise the GPU. Hence I'm trying guided_json, which is very fast, but doesn't work, as above.)

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[supersteves]

(Note that if you omit the reasoning command line arguments, structured outputs works correctly using json_format or guided_json, and is fast. Grammar is still slow. And of course the model won't "think" so is presumably less clever.)

[aarnphm]

are you using v1?

Right now v1 parser is not supported yet.

[supersteves]

@aarnphm 90% of my testing was with defaults which I think is v0.

(Although one of the docs seems to imply v1 is on by default. But I think that's wrong, because the logs only talked about v1 when I explicitly enabled it.)

I did find that v1 seemed to work with json schema, or at least, it didn't complain and the output happened to match. But this report is about docker vLLM default args (I guess, v0).

[aarnphm]

Can you explicitly set VLLM_USE_V1=0 for now? Until #16577 lands on next version of vLLM.

btw reasoning output still works for me on deepseek_r1 on v0 here.

[ympan0508]

Same issue for me

Serve QwQ-32B on 8x H100:

VLLM_USE_V1=0 CUDA_VISIBLE_DEVICES="0,1,2,3,4,5,6,7" NCCL_SOCKET_IFNAME="en,eth,em,bond" vllm serve /path/to/QwQ-32B --port 8765 --tensor_parallel 8 --served-model-name QwQ-32B --enable-reasoning --reasoning-parser deepseek_r1

Follow the script of https://docs.vllm.ai/en/latest/features/reasoning_outputs.html#structured-output

from pydantic import BaseModel
from openai import OpenAI

import os

os.environ['http_proxy'] = ""
os.environ['https_proxy'] = ""

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8765/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id


class People(BaseModel):
    name: str
    age: int


json_schema = People.model_json_schema()

prompt = ("Generate a JSON with the name and age of one random person.")
completion = client.chat.completions.create(
    model=model,
    messages=[{
        "role": "user",
        "content": prompt,
    }],
    extra_body={"guided_json": json_schema},
)
print("reasoning_content: ", completion.choices[0].message.reasoning_content)
print("content: ", completion.choices[0].message.content)

And it turns out to be:

reasoning_content:  {
  "name": "John Doe",
  "age": 30
}
content:  None

If the extra_body line is commented, all things work well:

reasoning_content:  Okay, the user wants me to generate a JSON object with the name and age of a random person. Let me start by understanding exactly what they need. They specified "name" and "age", so the JSON should have those two keys. The name should be a random string, and the age should be a random number within a reasonable range, say between 1 and 100......

content:  Here's an example of a JSON object with a random person's name and age:.......

[ympan0508]

Well, I might have been stupid :)

I was using vLLM==0.7.3, where reasoning + structured output might not have been implemented (see https://docs.vllm.ai/en/v0.7.3/features/reasoning_outputs.html)

Things work for me after upgrading to v0.8.4:

reasoning_content:  Okay, I need to generate a JSON object that includes the name and age of a random person. Let me start by thinking about how to approach this......

content:  {
    "name": "Liam",
    "age": 29
}

@supersteves You might check the vLLM version as well.

@aarnphm That said, I would suggest indicating in the documentation (https://docs.vllm.ai/en/stable/features) which version each feature was introduced in. This would help avoid confusion for users working with older releases.

[aarnphm]

That said, I would suggest indicating in the documentation (docs.vllm.ai/en/stable/features) which version each feature was introduced in. This would help avoid confusion for users working with older releases.

https://docs.vllm.ai/en/stable/features/reasoning_outputs.html ....

[supersteves]

Strange. I was using 0.8.3 (perhaps I should have posted that rather than the docker hub hash).
I'll re-post here when I get a chance to re-test with 0.8.4.

[supersteves]

I can confirm (0.8.5) that v0 works, v1 does not, as expected above. Apologies. I must have previously incorrectly assumed that v0 was the default when testing before.

Obviously, leave this closed.

[aarnphm]

Needs this to land in v1: #16577