-
-
Notifications
You must be signed in to change notification settings - Fork 11.3k
Add Olmo 3 reasoning parser #26054
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Olmo 3 reasoning parser #26054
Conversation
Signed-off-by: Luca Soldaini <luca@soldaini.net>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces a reasoning parser for Olmo 3 models. The implementation for non-streaming mode appears correct, and the test suite is a good start. However, the streaming implementation has a critical correctness issue that can lead to data loss when a stream ends with a partial reasoning tag. I've also identified some apparently unreachable code in the buffer processing logic. My review includes detailed comments on these issues.
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
|
Can you open a follow-up PR to add it to https://docs.vllm.ai/en/latest/features/reasoning_outputs.html? |
Signed-off-by: Luca Soldaini <luca@soldaini.net> Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>
Signed-off-by: Luca Soldaini <luca@soldaini.net> Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Luca Soldaini <luca@soldaini.net> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Signed-off-by: Luca Soldaini <luca@soldaini.net> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Signed-off-by: Luca Soldaini <luca@soldaini.net>
Purpose
This PR adds support for parsing reasoning traces for the Olmo 3 family of models. (support for models has been already merged in #24534).
Olmo models use
<think>and</think>string to bracket their reasoning traces; however, unlike Qwen or Deepseek families, the strings are not special tokens in the vocabulary, therefore requiring a new parser.Test Plan
This PR includes tests to verify that the parser behaves as expected. Existing tests should not be impacted.
Test Result
Run tests:
Test output:
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.