BUG: The gpt oss generated format cannot be processed correctly

[whfeLingYu]

I use OpenAIServerModel and ToolCallingAgent as the base. The model uses OpenAI's open source gpt oss. The extra parameters generated cannot be processed and the error is as follows

Error while generating output:
Error code: 400 - {'object': 'error', 'message': 'Expected 2 output messages (reasoning and final), but got 12.', 'type': 'BadRequestError', 'param': None, 'code': 400}

[jbejar]

Do you have any more detail about this bug that would be helpful to debug?

[bveiseh]

getting the same error. it seems smolagents is not compatible with how gpt-oss passes its thinking tokens

[also-good]

gpt-oss-120b with CodeAgent Frequent iterations without code

[yijun-lee]

Can you share a minimal reproducible example (a short script) that triggers this error on your side?

[jbejar]

@yijun-lee I can get a failing example with the smaller OSS model:

from smolagents import load_tool, CodeAgent, InferenceClientModel, DuckDuckGoSearchTool

# Import tool from Hub
image_generation_tool = load_tool("m-ric/text-to-image", trust_remote_code=True)


search_tool = DuckDuckGoSearchTool()

model = InferenceClientModel("openai/gpt-oss-20b") #Qwen/Qwen2.5-72B-Instruct works
# Initialize the agent with both tools
agent = CodeAgent(
    tools=[image_generation_tool, search_tool], model=model
)

# Run it!
result = agent.run(
    "Generate me a photo of the car that James bond drove in the latest movie.",
)

[suryabdev]

Posting some details I found on some other threads. I think there are two different errors

1. Expected 2 output messages (reasoning and final), but got
2. None output and issue with InferenceClientModel

For Error 1, It seems to be a provider specific issue with the harmony response format. From the docs

If you are not using gpt-oss directly but through an API or a provider like HuggingFace, Ollama, or vLLM, you will not have to be concerned about this as your inference solution will handle the formatting.

For vLLM, there is a long debugging thread, But it should be solved in the latest release because of PR vllm-project/vllm#23318
I searched github, sglang has the exact same implementation/bug sgl-project/sglang#8976

@whfeLingYu / @bveiseh If I'm not wrong both of you are using vLLM. Could you update vLLM and retry?

For Error 2, We'll have to debug a bit more to find the root cause

[bveiseh]

@suryabdev I am using litellm. still having this issue

[suryabdev]

@bveiseh LiteLLM forwards the request to a model/provider. It might be an issue with the provider you are using
Could you share the parameters you are using when initializing the LiteLLMModel?

Example

model = LiteLLMModel(
    model_id="anthropic/claude-4-sonnet-latest",
    temperature=0.2,
    api_key=os.environ["ANTHROPIC_API_KEY"]
)

[bveiseh]

@suryabdev Litellm normalizes all calls to be openai api compatible, so openai server should work. I tried as well with litellm server to be sure, and still got the same problem. its a shame because gpt oss is so good with other agent harnesses. Waiting for the day I can use it with smolagents.