[Frontend] Support reasoning content for deepseek r1

[gaocegege]

Fix #12468

Two CLI arguments are introduced for extensibility: --enable-reasoning and --reasoning-parser.

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B --enable-reasoning --reasoning-parser deepseek_r1

Test for full request:

from openai import OpenAI
client = OpenAI(
    base_url="http://100.104.240.69:8000/v1",
    api_key="token-abc123",
)

completion = client.chat.completions.create(
  model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
  messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
)

print("reasoning_content:", completion.choices[0].message.reasoning_content)
print("content", completion.choices[0].message.content)

Test for stream request:

from openai import OpenAI
client = OpenAI(api_key="<DeepSeek API Key>", base_url="http://100.104.240.69:8000/v1")

# Round 1
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
    messages=messages,
    stream=True
)

reasoning_content = ""
content = ""

for chunk in response:
    if chunk.choices[0].delta.reasoning_content:
        reasoning_content += chunk.choices[0].delta.reasoning_content
    else:
        content += chunk.choices[0].delta.content
    print(chunk)

print(f"Content: {content}")
print(f"Reasoning Content: {reasoning_content}")

These scripts are from https://api-docs.deepseek.com/guides/reasoning_model#api-example. However I encountered an issue with stream requests:

Traceback (most recent call last):
  File "main.py", line 16, in <module>
    if chunk.choices[0].delta.reasoning_content:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../site-packages/pydantic/main.py", line 828, in __getattr__
    raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
AttributeError: 'ChoiceDelta' object has no attribute 'reasoning_content'

Have to hack the openai python package to pass it. The RESTful API works well.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[DarkLight1337]

To make sure this is working correctly, can you add some tests to the codebase? Would also be great to add a section to the docs explaining its usage!

cc @mgoin @K-Mistele

[DarkLight1337]

Have to hack the openai python package to pass it. The RESTful API works well.

Yeah, I think we can just use requests library in the case where the API is incompatible with OpenAI spec.

[gaocegege]

To make sure this is working correctly, can you add some tests to the codebase? Would also be great to add a section to the docs explaining its usage!

cc @mgoin @K-Mistele

Will work on this.

[cksac]

How this works together with Guided/Structured Outputs?
Can we apply guided generation after reasoning content? i.e. apply grammar after specific token, i.e. </think>

[gaocegege]

@cksac Hi, please checkout #12468 (comment)

I don’t think they can work together. Also, the DeepSeek API doesn’t do function calls or structured output for DeepSeek R1, just FYI.

If you enable the feature with structured output, the output will be only the structured output.

[gaocegege]

I'm not certain if we can accept this. I used a tokenizer to mimic the streaming output process, which might require some memory.

[GeorgelPreput]

The DeepSeek tokenizer can be instantiated locally from the zip file they provide here: https://api-docs.deepseek.com/quick_start/token_usage

Though maybe this is worth upstreaming to Transformers?

[gaocegege]

I think we can use this:

    tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-R1")
    output = tokenizer.tokenize(param_dict["output"])

I used opt-125m to conserve memory during testing because I was unsure about the resources available on the test machine. If we can tolerate around 500MB of memory usage, we can use the R1 tokenizer directly.

[gaocegege]

@DarkLight1337 I added some unit tests, did not find how to add e2e tests, do we need add e2e cases for this feature? could you please guide me there?

[gaocegege]

Thanks for your time.

I think OpenAI still hides reasoning?

Yes, OpenAI hides the reasoning content.

vercel AI SDK just extract reasoning from the streamed tokens by looking for the XML tags

I hadn’t realized they support reasoning. Thanks for bringing that to my attention!

https://github.com/vercel/ai/blob/main/packages/ai/core/middleware/extract-reasoning-middleware.ts

I will take a look too.

[gaocegege]

Comments are addressed, please take a look.
Thanks for your time 😄

[mgoin]

Excellent work incorporating feedback and producing nicely documented code, LGTM! I'd like @DarkLight1337 or @K-Mistele to sign off as well before landing

[mgoin]

Why couldn't this just be choices=valid_reasoning_parsers?

[gaocegege]

Here’s the trick adapted from the tool parser argument.

https://github.com/vllm-project/vllm/blob/main/vllm/entrypoints/openai/cli_args.py#L216

I think it is to keep the consistency with other CLI arguments:

$ vllm serve --help
...
  --scheduling-policy {fcfs,priority}
...
--task {auto,generate,embedding,embed,classify,score,reward}
...
--tokenizer-mode {auto,slow,mistral}

[gaocegege]

The Vercel AI SDK has a design akin to this PR. It employs a regex to parse the entire output for synchronous generation requests, and we also use this method.

const regexp = new RegExp(`${openingTag}(.*?)${closingTag}`, 'gs');
      const matches = Array.from(text.matchAll(regexp));

      if (!matches.length) {
        return { text, ...rest };
      }

      const reasoning = matches.map(match => match[1]).join(separator);

For streaming requests, we also have a similar design that checks the start and end tokens in both the previous and delta text.

[gaocegege]

Thanks for your review 🥰

[K-Mistele]

Hey guys just now getting to this, glad to see it's been merged. For what it's worth, there have already been some discussions (ref. #11522 ) about refactoring tool parsers (possibly to use FSMs, which seems like it would be a good approach for managing tool reasoning parsing) instead of the current implementation which can tend to be buggy.

It seems like the direction we will want to move towards is having a single parser module for any given (supported) model that provides a unified interface for both tool parsing and reasoning parsing if one or both are supported by the model.

This would be more straightforward from a UX and a DX standpoint, and should result in cleaner code as well.

[gaocegege]

I think it's reasonable after a glance at the issue. Reasoning can be understood as a special tool, and they can share the implementation. During my implementation process, I did find the tool parser code to be quite messy.

[arunpatala]

I was wondering if json schema only for answer ( and not supressing the thinking tokens) is added in this PR?

[gaocegege]

I was wondering if json schema only for answer ( and not supressing the thinking tokens) is added in this PR?

It is not supported. ref #12468 (comment)