Skip to content

Conversation

@soldni
Copy link
Contributor

@soldni soldni commented Oct 1, 2025

Purpose

This PR adds support for parsing reasoning traces for the Olmo 3 family of models. (support for models has been already merged in #24534).

Olmo models use <think> and </think> string to bracket their reasoning traces; however, unlike Qwen or Deepseek families, the strings are not special tokens in the vocabulary, therefore requiring a new parser.

Test Plan

This PR includes tests to verify that the parser behaves as expected. Existing tests should not be impacted.

Test Result

Run tests:

pytest tests/reasoning/test_olmo3_reasoning_parser.py

Test output:

============================= test session starts ==============================
platform linux -- Python 3.12.11, pytest-8.4.2, pluggy-1.6.0
rootdir: /home/ec2-user/vllm
configfile: pyproject.toml
plugins: anyio-4.11.0, forked-1.6.0, cov-7.0.0, shard-0.1.2, asyncio-1.2.0, timeout-2.4.0, rerunfailures-16.0.1, mock-3.15.1, buildkite-test-collector-0.1.9, subtests-0.14.2, hypothesis-6.140.2, schemathesis-4.1.4, hydra-core-1.3.2, typeguard-4.4.4
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 14 items
Running 14 items in this shard

tests/reasoning/test_olmo3_reasoning_parser.py ..............            [100%]

============================== 14 passed in 3.50s ==============================

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the [Google Doc](https://docs.google.com/document/d/1YyVqrgX4gHTtrstbq8oWUImOyPCKSGnJ7xtTpmXzlRs/edit?tab=t.0).

Signed-off-by: Luca Soldaini <luca@soldaini.net>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a reasoning parser for Olmo 3 models. The implementation for non-streaming mode appears correct, and the test suite is a good start. However, the streaming implementation has a critical correctness issue that can lead to data loss when a stream ends with a partial reasoning tag. I've also identified some apparently unreachable code in the buffer processing logic. My review includes detailed comments on these issues.

soldni added 3 commits October 2, 2025 02:58
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
@mgoin mgoin added new-model Requests to new models ready ONLY add when PR is ready to merge/full CI is needed tool-calling labels Oct 3, 2025
@DarkLight1337 DarkLight1337 merged commit d0df145 into vllm-project:main Oct 4, 2025
47 checks passed
@DarkLight1337
Copy link
Member

Can you open a follow-up PR to add it to https://docs.vllm.ai/en/latest/features/reasoning_outputs.html?

tomeras91 pushed a commit to tomeras91/vllm that referenced this pull request Oct 6, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
karan pushed a commit to karan/vllm that referenced this pull request Oct 6, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>
southfreebird pushed a commit to southfreebird/vllm that referenced this pull request Oct 7, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new-model Requests to new models ready ONLY add when PR is ready to merge/full CI is needed tool-calling

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants