Using openai o*-mini generates an invalid tool's parameters schema.

[double-thinker]

Description

When using o1-mini or o3-mini model with function calls, strict: true is added vs gpt-4o that does not have strict flag. This leads to bugs when optional parameters are provided on the schema.

await generateText({
    model: openai("o3-mini"),
    prompt: "What is the weather here?",
    tools: {
      getWeather: tool({
        description: "Get the weather in a location",
        parameters: z.object({
        location: z
          .string()
          .optional()  // Note the optional
//    ...

Same bug happens with .default or .nullable. The root cause seems like when "strict": true is added the schema needs to change its typings as described here:

{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Retrieves current weather for the given location.",
        "strict": true,
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country e.g. Bogotá, Colombia"
                },
                "units": {
                    "type": ["string", "null"], // If strict is true, type must change this way. Currently, ai-sdk keeps "string" as type even with strict: true
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Units the temperature will be returned in."
                }
            },
            "required": ["location", "units"],
            "additionalProperties": false
        }
    }
}

As a workaround you can create a OpenAI client that removes strict flag:

const openaiWithWorkaround = createOpenAI({
  fetch: async (url, options) => {
    if (!options?.body) {
      return fetch(url, options);
    }

    // Parse the request body
    const body = JSON.parse(options.body as string);
    
    // Check if there are tools with functions
    if (body.tools?.length > 0) {
      body.tools = body.tools.map((tool: any) => {
        if (tool.type === 'function' && tool.function.strict) {
          // Remove the strict flag if present
          const { strict, ...functionWithoutStrict } = tool.function;
          return {
            ...tool,
            function: functionWithoutStrict
          };
        }
        return tool;
      });
    }

    // Create new options with modified body
    const newOptions = {
      ...options,
      body: JSON.stringify(body)
    };

    console.log(
      `Body ${JSON.stringify(
        JSON.parse((newOptions?.body as string) || "{}"),
        null,
        2
      )}`)

    // Make the actual fetch call
    return fetch(url, newOptions);
  },
});

Surprisingly, when using gpt-4o strict flag is not added vs when o*-mini is used. So I detected this bug when migrating to o3-mini

Code example

Reproduction and workaround here: https://gist.github.com/double-thinker/f60bde68cd5705a33288f2000eeec53d

AI provider

@ai-sdk/openai 1.1.9

Additional context

No response

[edenstrom]

Seems like structured outputs is activated by default for reasoning models:

ai/packages/openai/src/openai-chat-language-model.ts

Lines 59 to 64 in d0d13f9

	get supportsStructuredOutputs(): boolean {
	// enable structured outputs for reasoning models by default:
	// TODO in the next major version, remove this and always use json mode for models
	// that support structured outputs (blacklist other models)
	return this.settings.structuredOutputs ?? isReasoningModel(this.modelId);
	}

1. .optional() is incompatible with OpenAI structured outputs, and should be replaced with .nullable().
2. But issue remains that .nullable() doesn't work and still throws the error schema must have a \'type\' key.', for the properties with .nullable()

optional vs nullable documentation below:

OpenAI: https://platform.openai.com/docs/guides/structured-outputs#supported-schemas

Vercel AI SDK: https://sdk.vercel.ai/providers/ai-sdk-providers/openai#structured-outputs

[lgrammel]

You can opt out of structured outputs by changing the structuredOutputs setting to false for reasoning models (default to true).

[double-thinker]

@lgrammel but as @edenstrom mentioned, nullable does not work either, so I think the bug is still present. If this will not be fixed, I think a mention in the docs would be worthwhile.

[hopkins385]

@lgrammel can you explain why structuredOutputs is true per default only for the reasoning models?

I'd prefer to keep the same default behaviour for all models.
If for all models structuredOutputs=true fine, but if we start to mix it, it becomes quite hard to maintain.

imho, the default behaviour should be structuredOutputs=false

[lgrammel]

@hopkins385 the goal is to move it to true for all models, but that needs to wait for 5.0 for backwards compat. However, since reasoning models are new, we can already enable it for those.

[hopkins385]

@lgrammel why?

[lgrammel]

1. Added a test for the nullable behavior. It is as requested by the OpenAI spec: chore (ui): add test for zod schema nullable #4712 - so this can't be the issue here

2. There are rumors that OpenAI behaves differently for gpt-4o for o3-mini when it comes to structured outputs: https://x.com/dzhng/status/1886945642242789812 (so the issue might be on the OpenAI side)

3. You can disable structured outputs on the model: https://sdk.vercel.ai/providers/ai-sdk-providers/openai#chat-models (structuredOutputs setting)

[vitorbal]

@lgrammel is there any docs/issue on why you want to move to have structured outputs enabled by default? Seems like that could be a bit of a footgun in some cases. Would love to read more about the reasoning behind that decision!

[lgrammel]

@vitorbal the goal was always to make structured outputs enabled by default; for all models, because it enables more robust outputs. it only default to false for other models for backwards compat for now.

[hopkins385]

Still not clear to me why structured outputs should be enabled by default as it is not the default behavior of the models. I'd propose to adhere to the default behavior of the model providers. Also we should consider that not all models and/or providers even support structured outputs.

Can you share more details on why the ai sdk plans to defer from the defaults of all model providers?

PS: structured outputs needs to be configured separately and is just useful for very specific use-cases, but not for the standard chat-completion, afaik.

[lgrammel]

@hopkins385 contrast the 2 options:

1. structured outputs are opt-in: sometimes the model will return bad completions that cannot be parsed, leading to unforeseen errors in production
2. structured outputs are opt-out: you discover immediately when your schema is incompatible (provider error during dev time) and you can opt-out. by default, the model will always return correct completions, preventing parsing errors in production. you know when you opted out and created a risk

that's why i think opt-out is preferrable

[hopkins385]

By having the opt-out option you are assuming everyone prefers to configure a JSON schema and parse the output of the llm. Which means the main use-case is not chat-completion. But afaik this is not the case as the main use-case is just forwarding the tokens to the client (e.g. chat scenario).

As openai writes in their docs:
"Structured Outputs is a feature that ensures the model will always generate responses that adhere to your supplied JSON Schema, so you don't need to worry about the model omitting a required key, or hallucinating an invalid enum value." source

I think "features" should always be opt-in.

[lgrammel]

@hopkins385 noted that you think that. also noted that you do not know that the ai sdk only sets structured outputs if you actually supply a schema

[hopkins385]

@lgrammel constructive feedback please. As you can see/read the confusion is already there and I am not alone with it.