fixed reasoning streaming with tool_choice="required"

[ExtReMLapin]

Purpose

fixed reasoning not being sent to client when tool_choice="required"

closes #14429

Test Plan

Added one test to ensure it's returned in streamed data pytest ./tests/entrypoints/openai/test_completion_with_function_calling.py

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[ExtReMLapin]

Tested on multiple qwen models + tool

• Qwen 3 with reasoning
• Qwen 3 with reasoning disabled
• Qwen 2.5

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ExtReMLapin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[ExtReMLapin]

@DarkLight1337 @heheda12345
@simon-mo

I'm not sure exactly who to ping to get it reviewed

[DarkLight1337]

cc @aarnphm @chaunceyjiang

[chaunceyjiang]

reasoning not being sent to client when tool_choice="required"

Could you provide a reproduction step?

The combination of stream + enable_thinking + required has been continuously tested in e2e.

https://github.com/vllm-project/vllm/blob/main/tests/entrypoints/openai/test_completion_with_function_calling.py#L165-L177

[ExtReMLapin]

@chaunceyjiang there is no assert/check/test in the stream mode for reasoning

https://github.com/vllm-project/vllm/blob/main/tests/entrypoints/openai/test_completion_with_function_calling.py#L218

master/HEAD :

(See it directly goes into tool call , something like 10 seconds between first message and tool call start)

query.js

---

This branch :

[ExtReMLapin]

Also this PR cover both forced reasoning models (like Qwen3 2507 that doesn't output opening reasoning tag) and original ones that outputs both opening and closing reasoning tags

[chaunceyjiang]

Good catch.

Can you add a check for this here: https://github.com/vllm-project/vllm/blob/main/tests/entrypoints/openai/test_completion_with_function_calling.py#L165-L177?

[ExtReMLapin]

Got it for the changes.

In the tests i'm having a weird issue where

        output = []
        reasoning = []
        async for chunk in output_stream:
            if chunk.choices:
                if enable_thinking and chunk.choices[0].delta.reasoning_content:
                    reasoning.append(chunk.choices[0].delta.reasoning_content)
                if chunk.choices[0].delta.tool_calls:
                    output.extend(chunk.choices[0].delta.tool_calls)

        assert len(output) > 0
        if enable_thinking:
            assert len(reasoning) > 0

Doesn't work because the openai class doesn't have this param, and I don't understand why it doesn't error with the non stream part.

So instead I moved to if enable_thinking and getattr(chunk.choices[0].delta, "reasoning_content", None): not sure if it's good.

[ExtReMLapin]

And something's broken with non reasoning models so i'll fix it when I get back from vacations

[ExtReMLapin]

/gemini review

[gemini-code-assist]

Code Review

This pull request fixes an issue where reasoning content was not streamed correctly when tool_choice="required". The fix involves using the correct streaming-aware function for extracting reasoning content. The associated test is also updated to verify this behavior.

My review focuses on the maintainability of the fix. While the fix is correct, it introduces code duplication for handling reasoning streaming across different tool_choice scenarios. I've suggested refactoring this duplicated logic into a helper function to improve code clarity and reduce the risk of future inconsistencies.

[ExtReMLapin]

Considering the precommit warning linked to the values of reasoning_end_arr in

        if tool_choice_auto or self.reasoning_parser:
            # These are only required in "auto" tool choice case
            all_previous_token_ids = [[]] * num_choices
            # For reasoning parser and tool call all enabled
            added_content_delta_arr = [False] * num_choices
            reasoning_end_arr = [False] * num_choices
        else:
            all_previous_token_ids = None
            reasoning_end_arr = None

Would you be fine with reasoning_end_arr = [False] * num_choices being initialized either way outside of the if ? @chaunceyjiang

[chaunceyjiang]

Would you be fine with reasoning_end_arr = [False] * num_choices being initialized either way outside of the if ? @chaunceyjiang

I haven’t reviewed your PR carefully yet, but my understanding is that reasoning_end_arr should only be used when if self.reasoning_parser is True.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ExtReMLapin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[ExtReMLapin]

Could it be possible to have a merge on that ? It's honestly not that complicated to see it's fixed and as of right now with reasoning model it's not possible to know what's going on if it's a long generation loop, all you'll see in the server console is it generated token (Avg generation throughput: 51.6 tokens/s,) but you can't know if it's actualy legitimate reasonining tokens or stuck in an infinite loop.

And obviously with non streamer queries, you don't know anything, you don't even know if you went past the reasoning part.

cc @chaunceyjiang

[chaunceyjiang]

Thanks~

[chaunceyjiang]

Hi @ExtReMLapin, there are currently some issues with the CI on the main branch. Let's wait for them to be fixed before proceeding with this PR.

[ExtReMLapin]

Hooray !