[Refactor][Frontend] Keep all logic about reasoning into one class #14428

gaocegege · 2025-03-07T10:02:46Z

This is a quick refactor to get us ready to tackle a few issues like #14399, #14170, #14335, #14511, #14429 and #14490

Right now, we’ve got two separate abstractions of handling reasoning—one for OpenAI chat and another for structured outputs.

But if we want to fix #14399, which involves adding checks for reasoning tokens in the stopChecker, we’d probably end up with even more abstractions.

To keep things clean and avoid a mess, we’re merging these into one unified approach.

github-actions · 2025-03-07T10:02:55Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

gaocegege · 2025-03-07T10:12:03Z

/cc @alex-jw-brooks

gaocegege · 2025-03-10T02:53:56Z

I hope this can be merged after PR #14202 is finalized, as there might be some conflicts. I’m happy to resolve those conflicts directly in this PR. I’ll wait for @alex-jw-brooks to confirm if this approach works.

aarnphm

Given that this is strictly frontend, I think we should keep it under entrypoint.

gaocegege · 2025-03-17T01:32:35Z

Given that this is strictly frontend, I think we should keep it under entrypoint.

Thanks for your review.

It’s used in model_executor.guided_decoding. is it okay to import entrypoints there?

aarnphm · 2025-03-17T02:07:06Z

It’s used in model_executor.guided_decoding. is it okay to import entrypoints there?

hmm, thinking more about this, it is probably ok to add a vllm.reasoning (iirc we don't do any import in entrypoints, but that is not guarantee)

mergify · 2025-03-19T15:01:50Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @gaocegege.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

aarnphm

Just some small comments, and a rename preference to vllm.reasoning, otherwise LGTM.

examples/online_serving/openai_chat_completion_with_reasoning.py

vllm/engine/arg_utils.py

vllm/reasoning_parser/abs_reasoning_parsers.py

aarnphm

ok, final round of review. just final few thoughts

vllm/reasoning/abs_reasoning_parsers.py

aarnphm · 2025-03-27T05:23:29Z

vllm/reasoning/abs_reasoning_parsers.py

-                         force: bool = True) -> None:
+    def _register_module(
+        cls,
+        module: type,


Suggested change

module: type,

module: type[Any],

? or does it take in a type variable ? iirc we can also pass in a generic type here?

This is copied from the tool parser https://github.com/vllm-project/vllm/blob/main/vllm/entrypoints/openai/tool_parsers/abstract_tool_parser.py#L99

Should we change it too?

let's do it in a separate PR then.

OK, I created an issue to keep track. #15658

aarnphm · 2025-03-27T05:24:45Z

vllm/reasoning/abs_reasoning_parsers.py

+        cls,
+        module: type,
+        module_name: Optional[Union[str, list[str]]] = None,
+        force: bool = True,


nit: This seems to be the strict behaviour. I would prefer renaming this variable to strict here.

also iiuc this is internal anyways, should we just make it strict by default?

I dont' really see the reason for user overriding this any time soon (given that we are maintaining this anw)?

nit: for register_module I think would be nice if we can transform the target class into a dataclass, so that we can enforce start_token and end_token string

And setup start_token_id and end_token_id accordingly based on the vocab

ditto. This is copied from the tool parser, maybe we could have another PR to change both of them?

aarnphm · 2025-03-27T05:28:41Z

vllm/reasoning/deepseek_r1_reasoning_parser.py

+    start_token_id: int
+    end_token_id: int
+
+    start_token: str = "<think>"
+    end_token: str = "</think>"


above for logics.

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

gaocegege · 2025-03-28T00:57:27Z

@DarkLight1337 @mgoin @simon-mo Could you please help merge this?

gaocegege · 2025-03-28T07:19:41Z

The entrypoints test timed out. Could we merge it manually?

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai> Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai> Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai>

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai> Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai>

mergify bot added documentation Improvements or additions to documentation frontend structured-output labels Mar 7, 2025

gaocegege marked this pull request as ready for review March 7, 2025 10:10

gaocegege requested review from DarkLight1337, alexm-redhat, comaniac, mgoin, njhill, robertgshaw2-redhat, simon-mo, youkaichao and zhuohan123 as code owners March 7, 2025 10:10

gaocegege force-pushed the refactor branch from 1e69169 to 1b1ad93 Compare March 7, 2025 10:25

gaocegege mentioned this pull request Mar 9, 2025

[Frontend] Support tool calling and reasoning parser #14511

Merged

gaocegege force-pushed the refactor branch from 1b1ad93 to 826d1da Compare March 10, 2025 02:09

gaocegege requested a review from russellb as a code owner March 10, 2025 02:09

gaocegege force-pushed the refactor branch from 826d1da to 9040b0c Compare March 10, 2025 07:52

gaocegege mentioned this pull request Mar 10, 2025

[Frontend] Skip stop in reasoning content #14550

Merged

gaocegege changed the title ~~[Refactor][Reasoning] Keep all logic about reasoning into one class~~ [Refactor][Frontend] Keep all logic about reasoning into one class Mar 10, 2025

aarnphm requested changes Mar 16, 2025

View reviewed changes

mergify bot added the needs-rebase label Mar 19, 2025

aarnphm approved these changes Mar 19, 2025

View reviewed changes

theobjectivedad mentioned this pull request Mar 23, 2025

[Feature]: Add support for reusable subschemas in tool requests (PydanticAI) #15035

Closed

1 task

aarnphm approved these changes Mar 27, 2025

View reviewed changes

aarnphm removed request for alexm-redhat, comaniac, njhill, robertgshaw2-redhat, simon-mo, youkaichao and zhuohan123 March 27, 2025 05:29

Remove pass

18024cf

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

gaocegege mentioned this pull request Mar 28, 2025

[Feature]: Refactor the logic in tool parser manager and reasoning parser manager #15658

Open

1 task

DarkLight1337 approved these changes Mar 28, 2025

View reviewed changes

DarkLight1337 enabled auto-merge (squash) March 28, 2025 02:49

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 28, 2025

vllm-bot merged commit 32b14ba into vllm-project:main Mar 28, 2025
45 of 47 checks passed

gaocegege deleted the refactor branch March 28, 2025 07:56

This was referenced Mar 28, 2025

[Feature]: reasoning outputs in structured outputs in v1 #14727

Closed

[Bugfix][Frontend] Eliminate regex based check in reasoning full generator #14821

Merged

lulmer pushed a commit to lulmer/vllm that referenced this pull request Apr 7, 2025

[Refactor][Frontend] Keep all logic about reasoning into one class (v…

7ff23c2

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai> Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>

ckhordiasma mentioned this pull request Apr 17, 2025

[do not merge] pr test for nm changes into 2.20 red-hat-data-services/vllm#107

Closed

lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Apr 29, 2025

[Refactor][Frontend] Keep all logic about reasoning into one class (v…

068eec9

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai>

shreyankg pushed a commit to shreyankg/vllm that referenced this pull request May 3, 2025

[Refactor][Frontend] Keep all logic about reasoning into one class (v…

35dfffa

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai>

reidliu41 mentioned this pull request May 17, 2025

[doc] update reasoning doc #18306

Merged

huiqiwa pushed a commit to huiqiwa/vllm-fork that referenced this pull request Oct 21, 2025

[Refactor][Frontend] Keep all logic about reasoning into one class (v…

bdb1d9e

…llm-project#14428) Signed-off-by: Ce Gao <cegao@tensorchord.ai>

Uh oh!

Uh oh!

[Refactor][Frontend] Keep all logic about reasoning into one class #14428

[Refactor][Frontend] Keep all logic about reasoning into one class #14428

Uh oh!

Conversation

gaocegege commented Mar 7, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 7, 2025

Uh oh!

gaocegege commented Mar 7, 2025

Uh oh!

gaocegege commented Mar 10, 2025

Uh oh!

aarnphm left a comment

Choose a reason for hiding this comment

Uh oh!

gaocegege commented Mar 17, 2025

Uh oh!

aarnphm commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify bot commented Mar 19, 2025

Uh oh!

aarnphm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aarnphm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gaocegege commented Mar 28, 2025

Uh oh!

gaocegege commented Mar 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gaocegege commented Mar 7, 2025 •

edited by github-actions bot

Loading

aarnphm commented Mar 17, 2025 •

edited

Loading