Add support for OpenAI o1

[dmtran-g]

Feature Description

OpenAI has just released o1-2024-12-17. See https://platform.openai.com/docs/models/gpt-4o#o1

I tried to use o1-2024-12-17 with streamText, but ran into the following error:

"type": "APICallError",
"message": "Unsupported value: 'stream' does not support true with this model. Supported values are: false.",

Note: according to https://platform.openai.com/docs/guides/reasoning, o1 supports "developer messages" (whereas o1-preview didn't support system messages):

Developer messages are the new system messages: Starting with o1-2024-12-17, o1 models support developer messages rather than system messages, to align with the chain of command behavior described in the model spec. Learn more.

Use Cases

No response

Additional context

No response

[dmtran-g]

Actually, it looks like the above error comes from the OpenAI API itself.

According to https://platform.openai.com/docs/guides/reasoning#limitations

Streaming support not available in the REST API

Edit: o1-preview supports streaming, but it looks like o1 doesn't support it yet

[lgrammel]

Released in @ai-sdk/openai@1.0.9

[dmtran-g]

Hi @lgrammel

Thank you so much for quickly adding support for o1!

I took a look at the changes, it looks like "developer" messages are not yet supported for o1.
Should I create a new issue for this?

According to https://platform.openai.com/docs/guides/reasoning

Developer messages are the new system messages: Starting with o1-2024-12-17, o1 models support developer messages rather than system messages, to align with the chain of command behavior described in the model spec. Learn more.

[lgrammel]

You don't need them, OpenAI will translate system messages accordingly (it's just a rename)

[dmtran-g]

You don't need them, OpenAI will translate system messages accordingly (it's just a rename)

I see, thanks for the explanation!

[kdawgwilk]

We are still seeing errors with o1-preview and using system messages so i think the SDK still needs to map them?

AI_APICallError: Unsupported value: 'messages[0].role' does not support 'system' with this model.

[dmtran-g]

We are still seeing errors with o1-preview and using system messages

o1-preview doesn't support system messages, but o1 does (though I haven't tried it myself yet, as o1 doesn't support streaming yet)

[lgrammel]

Planning to automatically convert system messages to developer messages for openai reasoning models

[mutewinter]

@lgrammel the default behavior that is currently shipped removes the system messages silently. I'm not even seeing any warnings when this happens. I've checked the result object and it's undefined. I'm using streamText.

ai/packages/openai/src/openai-chat-language-model.ts

Lines 955 to 972 in 0118fa7

	const reasoningModels = {
	'o1-mini': {
	systemMessageMode: 'remove',
	simulateStreamingByDefault: false,
	},
	'o1-mini-2024-09-12': {
	systemMessageMode: 'remove',
	simulateStreamingByDefault: false,
	},
	'o1-preview': {
	systemMessageMode: 'remove',
	simulateStreamingByDefault: false,
	},
	'o1-preview-2024-09-12': {
	systemMessageMode: 'remove',
	simulateStreamingByDefault: false,
	},
	} as const;

Versions I'm running

    "@ai-sdk/openai": "^1.0.19",
    "ai": "^4.0.37",

Certainly the default should not be to drop system messages silently. I feel like this should throw an error with how unexpected this behavior is. OpenRouter seems to be silently converting system messages to developer messages, which feels like another option.

[lgrammel]

@mutewinter

a) o1-mini and o1-preview rejected both developer messages and system messages when I last check; therefore stripping is required
b) warnings are available, check the warnings property of the response
c) "Certainly"

[mutewinter]

I tried a few raw requests to OpenAI and you're absolutely right that they are currently erroring on "developer" and "system" messages for o1-mini. Even though their docs indicate they support developer messages for the o1 family of models:

Developer messages are the new system messages: Starting with o1-2024-12-17, o1 models support developer messages rather than system messages, to align with the chain of command behavior described in the model spec. Learn more.

I feel the default behavior of the AI SDK to drop system messages is dangerous and unexpected (even if warnings are returned).

Possible Fixes

a) Convert system messages to user or assistant messages (this feels like it could create a lot of downstream issues, but would at least retain some instruction to the LLM)
b) Throw a runtime error
c) Send role: "developer" messages to OpenAI for o1-family models and allow them to return an API error until they support role: "developer" someday