Streaming chunks increment choice index but shouldn't

[joergjo]

v0.7.117 has fixed #221, which caused the id field to be incremented. But there is another field that is incremented in every chunk which shouldn't: choice.index. Assuming we call the chat completion endpoint with n=1 (only one choice), index must be 0 in every chunk (the chunks are all part of the chat completion response).

{"model":"qwen2.5-1.5b-instruct-generic-gpu:3","choices":[{"delta":{"role":"assistant","content":"Wh","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"message":{"role":"assistant","content":"Wh","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"index":0,"finish_reason":null,"finish_details":null,"logprobs":null}],"usage":null,"system_fingerprint":null,"service_tier":null,"created":1758888848,"CreatedAt":"2025-09-26T12:14:08+00:00","id":"chat.id.1","StreamEvent":null,"IsDelta":false,"Successful":true,"error":null,"HttpStatusCode":0,"HeaderValues":null,"object":"chat.completion.chunk"}

{"model":"qwen2.5-1.5b-instruct-generic-gpu:3","choices":[{"delta":{"role":"assistant","content":"isper","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"message":{"role":"assistant","content":"isper","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"index":1,"finish_reason":null,"finish_details":null,"logprobs":null}],"usage":null,"system_fingerprint":null,"service_tier":null,"created":1758888848,"CreatedAt":"2025-09-26T12:14:08+00:00","id":"chat.id.1","StreamEvent":null,"IsDelta":false,"Successful":true,"error":null,"HttpStatusCode":0,"HeaderValues":null,"object":"chat.completion.chunk"}

{"model":"qwen2.5-1.5b-instruct-generic-gpu:3","choices":[{"delta":{"role":"assistant","content":"ing","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"message":{"role":"assistant","content":"ing","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"index":2,"finish_reason":null,"finish_details":null,"logprobs":null}],"usage":null,"system_fingerprint":null,"service_tier":null,"created":1758888848,"CreatedAt":"2025-09-26T12:14:08+00:00","id":"chat.id.1","StreamEvent":null,"IsDelta":false,"Successful":true,"error":null,"HttpStatusCode":0,"HeaderValues":null,"object":"chat.completion.chunk"}

{"model":"qwen2.5-1.5b-instruct-generic-gpu:3","choices":[{"delta":{"role":"assistant","content":" wind","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"message":{"role":"assistant","content":" wind","name":null,"tool_call_id":null,"function_call":null,"tool_calls":[]},"index":3,"finish_reason":null,"finish_details":null,"logprobs":null}],"usage":null,"system_fingerprint":null,"service_tier":null,"created":1758888848,"CreatedAt":"2025-09-26T12:14:08+00:00","id":"chat.id.1","StreamEvent":null,"IsDelta":false,"Successful":true,"error":null,"HttpStatusCode":0,"HeaderValues":null,"object":"chat.completion.chunk"}

Here is an example from the OpenAI REST API spec:

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-4o-mini", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-4o-mini", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{"content":"Hello"},"logprobs":null,"finish_reason":null}]}

...

{"id":"chatcmpl-123","object":"chat.completion.chunk","created":1694268190,"model":"gpt-4o-mini", "system_fingerprint": "fp_44709d6fcb", "choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

This causes the current Go OpenAI SDK (2.7.0) to break when handling streaming response from Foundry Local.

[natke]

Thanks for reporting this @joergjo. We are looking into it