Skip to content

Conversation

@qandrew
Copy link
Contributor

@qandrew qandrew commented Sep 16, 2025

Purpose

I noticed that While a ResponseContentPartCreatedEvent is added for reasoning items, ResponseContentPartDoneEvent isn't there. There's actually an underlying issue with openai python package, and I've filed an issue here: openai/openai-python#2634

It seems like the best path forward is to implement ResponseReasoningPartAddedEvent, ResponseReasoningPartDoneEvent in vLLM as oAI may take a while to add to their python library. When that is done, we can remove these types in vLLM.

Test Plan

added a unit test

in addition, we tested manually below:

server

(gpt_oss_edit) [axia@devvm30969.cln0 /data/users/axia/gitrepos/vllm (andrew/gpt-oss-reasoning-part)]$ CUDA_VISIBLE_DEVICES=2,3 with-proxy vllm serve "/data/users/axia/checkpoints/gpt-oss-120b" -tp 2 --port 20001

client

curl http://localhost:20001/v1/responses   -H "Content-Type: application/json"   -N   -d '{
    "model": "/data/users/axia/checkpoints/gpt-oss-120b",
    "input": [
        {
            "role": "user",
            "content": "Hello."
        }
    ],
    "temperature": 0.7,
    "max_output_tokens": 256,
    "stream": true
}'  

results (note response.reasoning_part.added, response.reasoning_part.done)

event: response.created
data: {"response":{"id":"resp_a35e27c4bf004ac08988e16d28d8b4d8","created_at":1758138055.0,"error":null,"incomplete_details":null,"instructions":null,"metadata":null,"model":"/data/users/axia/checkpoints/gpt-oss-120b","object":"response","output":[],"parallel_tool_calls":true,"temperature":0.7,"tool_choice":"auto","tools":[],"top_p":1.0,"background":false,"conversation":null,"max_output_tokens":256,"max_tool_calls":null,"previous_response_id":null,"prompt":null,"prompt_cache_key":null,"reasoning":null,"safety_identifier":null,"service_tier":"auto","status":"in_progress","text":null,"top_logprobs":null,"truncation":"disabled","usage":null,"user":null},"sequence_number":0,"type":"response.created"}

event: response.in_progress
data: {"response":{"id":"resp_a35e27c4bf004ac08988e16d28d8b4d8","created_at":1758138055.0,"error":null,"incomplete_details":null,"instructions":null,"metadata":null,"model":"/data/users/axia/checkpoints/gpt-oss-120b","object":"response","output":[],"parallel_tool_calls":true,"temperature":0.7,"tool_choice":"auto","tools":[],"top_p":1.0,"background":false,"conversation":null,"max_output_tokens":256,"max_tool_calls":null,"previous_response_id":null,"prompt":null,"prompt_cache_key":null,"reasoning":null,"safety_identifier":null,"service_tier":"auto","status":"in_progress","text":null,"top_logprobs":null,"truncation":"disabled","usage":null,"user":null},"sequence_number":1,"type":"response.in_progress"}

event: response.output_item.added
data: {"item":{"id":"msg_b6af18840c294ba1aa494d82e5892bef","summary":[],"type":"reasoning","content":null,"encrypted_content":null,"status":"in_progress"},"output_index":0,"sequence_number":2,"type":"response.output_item.added"}

event: response.reasoning_part.added
data: {"content_index":0,"item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"part":{"text":"","type":"reasoning_text"},"sequence_number":3,"type":"response.reasoning_part.added"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":"The","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":4,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" user","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":5,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" says","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":6,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" \"","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":7,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":"Hello","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":8,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":".\"","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":9,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" Probably","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":10,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" they","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":11,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" want","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":12,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" a","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":13,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" greeting","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":14,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":".","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":15,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" I","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":16,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" should","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":17,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" respond","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":18,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" politely","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":19,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":",","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":20,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" ask","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":21,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" how","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":22,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" can","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":23,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" I","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":24,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" help","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":25,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":".","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":26,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" Also","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":27,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" maybe","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":28,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" ask","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":29,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" if","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":30,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" they'd","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":31,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" like","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":32,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" to","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":33,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" chat","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":34,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":".","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":35,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" Keep","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":36,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":" friendly","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":37,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.delta
data: {"content_index":0,"delta":".","item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":0,"sequence_number":38,"type":"response.reasoning_text.delta"}

event: response.reasoning_text.done
data: {"content_index":0,"item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":1,"sequence_number":39,"text":"The user says \"Hello.\" Probably they want a greeting. I should respond politely, ask how can I help. Also maybe ask if they'd like to chat. Keep friendly.","type":"response.reasoning_text.done"}

event: response.reasoning_part.done
data: {"content_index":0,"item_id":"msg_b6af18840c294ba1aa494d82e5892bef","output_index":1,"part":{"text":"The user says \"Hello.\" Probably they want a greeting. I should respond politely, ask how can I help. Also maybe ask if they'd like to chat. Keep friendly.","type":"reasoning_text"},"sequence_number":40,"type":"response.reasoning_part.done"}

event: response.output_item.done
data: {"item":{"id":"msg_b6af18840c294ba1aa494d82e5892bef","summary":[],"type":"reasoning","content":[{"text":"The user says \"Hello.\" Probably they want a greeting. I should respond politely, ask how can I help. Also maybe ask if they'd like to chat. Keep friendly.","type":"reasoning_text"}],"encrypted_content":null,"status":"completed"},"output_index":1,"sequence_number":41,"type":"response.output_item.done"}

event: response.output_item.added
data: {"item":{"id":"msg_16f9b2b482fb428abc4456f279a06d8b","content":[],"role":"assistant","status":"in_progress","type":"message"},"output_index":1,"sequence_number":42,"type":"response.output_item.added"}

event: response.content_part.added
data: {"content_index":1,"item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","output_index":1,"part":{"annotations":[],"text":"","type":"output_text","logprobs":[]},"sequence_number":43,"type":"response.content_part.added"}

event: response.output_text.delta
data: {"content_index":1,"delta":"Hello","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":44,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":"!","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":45,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":" 👋","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":46,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":" How","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":47,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":" can","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":48,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":" I","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":49,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":" assist","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":50,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":" you","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":51,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":" today","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":52,"type":"response.output_text.delta"}

event: response.output_text.delta
data: {"content_index":1,"delta":"?","item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":1,"sequence_number":53,"type":"response.output_text.delta"}

event: response.output_text.done
data: {"content_index":1,"item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","logprobs":[],"output_index":2,"sequence_number":54,"text":"Hello! 👋 How can I assist you today?","type":"response.output_text.done"}

event: response.content_part.done
data: {"content_index":1,"item_id":"msg_16f9b2b482fb428abc4456f279a06d8b","output_index":2,"part":{"annotations":[],"text":"Hello! 👋 How can I assist you today?","type":"output_text","logprobs":null},"sequence_number":55,"type":"response.content_part.done"}

event: response.output_item.done
data: {"item":{"id":"msg_16f9b2b482fb428abc4456f279a06d8b","content":[{"annotations":[],"text":"Hello! 👋 How can I assist you today?","type":"output_text","logprobs":null}],"role":"assistant","status":"completed","type":"message"},"output_index":2,"sequence_number":56,"type":"response.output_item.done"}

event: response.completed
data: {"response":{"id":"resp_a35e27c4bf004ac08988e16d28d8b4d8","created_at":1758138055.0,"error":null,"incomplete_details":null,"instructions":null,"metadata":null,"model":"/data/users/axia/checkpoints/gpt-oss-120b","object":"response","output":[{"id":"rs_5440ad6878f340b18aa80855155a7e8d","summary":[],"type":"reasoning","content":[{"text":"The user says \"Hello.\" Probably they want a greeting. I should respond politely, ask how can I help. Also maybe ask if they'd like to chat. Keep friendly.","type":"reasoning_text"}],"encrypted_content":null,"status":null},{"id":"msg_7bc464fb42c3490cb1b0dffb709bc14a","content":[{"annotations":[],"text":"Hello! 👋 How can I assist you today?","type":"output_text","logprobs":null}],"role":"assistant","status":"completed","type":"message"}],"parallel_tool_calls":true,"temperature":0.7,"tool_choice":"auto","tools":[],"top_p":1.0,"background":false,"conversation":null,"max_output_tokens":256,"max_tool_calls":null,"previous_response_id":null,"prompt":null,"prompt_cache_key":null,"reasoning":null,"safety_identifier":null,"service_tier":"auto","status":"completed","text":null,"top_logprobs":null,"truncation":"disabled","usage":{"input_tokens":67,"input_tokens_details":{"cached_tokens":64},"output_tokens":56,"output_tokens_details":{"reasoning_tokens":36,"tool_output_tokens":0},"total_tokens":123},"user":null},"sequence_number":57,"type":"response.completed"}

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@qandrew qandrew force-pushed the andrew/gpt-oss-reasoning-part branch from 71694d7 to bad718f Compare September 17, 2025 22:22
@qandrew qandrew marked this pull request as ready for review September 17, 2025 22:23
@qandrew
Copy link
Contributor Author

qandrew commented Sep 17, 2025

cc @yeqcharlotte , @heheda12345 ready for review!

Signed-off-by: Andrew Xia <axia@meta.com>
@qandrew qandrew force-pushed the andrew/gpt-oss-reasoning-part branch from b39727e to ed737ab Compare September 17, 2025 22:44
@qandrew qandrew changed the title [gpt-oss] Fix ResponseContentPartDoneEvent in stream output [gpt-oss] Add ResponseReasoningPartAddedEvent, ResponseReasoningPartTypesEvent for streaming Sep 17, 2025
@qandrew qandrew changed the title [gpt-oss] Add ResponseReasoningPartAddedEvent, ResponseReasoningPartTypesEvent for streaming [gpt-oss] Add ResponseReasoningPartAddedEvent, ResponseReasoningPartDoneEvent for streaming Sep 17, 2025
Copy link
Collaborator

@yeqcharlotte yeqcharlotte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! thanks for the change!

@github-project-automation github-project-automation bot moved this from To Triage to Ready in gpt-oss Issues & Enhancements Sep 18, 2025
@yeqcharlotte yeqcharlotte enabled auto-merge (squash) September 18, 2025 07:55
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 18, 2025
@yeqcharlotte
Copy link
Collaborator

@qandrew test failure is related

Signed-off-by: Andrew Xia <axia@meta.com>
@yeqcharlotte yeqcharlotte merged commit 6d8246a into vllm-project:main Sep 19, 2025
43 checks passed
ywang96 pushed a commit to ywang96/vllm that referenced this pull request Sep 19, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
debroy-rh pushed a commit to debroy-rh/vllm that referenced this pull request Sep 19, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
charlifu pushed a commit to ROCm/vllm that referenced this pull request Sep 25, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: charlifu <charlifu@amd.com>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
…oneEvent for streaming (vllm-project#24938)

Signed-off-by: Andrew Xia <axia@meta.com>
Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

frontend gpt-oss Related to GPT-OSS models ready ONLY add when PR is ready to merge/full CI is needed

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants