ChatOpenAI with o3-mini fails when pulling a prompt with model from Langsmith using pull

[nikhilshinday]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import * as hub from "langchain/hub"; // I can also reproduce this with "langchain/hub/node"

const template = await hub.pull<
  ChatPromptTemplate<{
    report: {
      name: string;
      description: string;
      section: {
        type: string;
        description: string;
      };
    };
    url: string;
    data: string;
  }>
>("summary__prompt", { includeModel: true }); // model is o3-mini, see screenshot

const summary = (
  await template.invoke({
    report: input.report,
    url: "https://app-staging.buildbetter.app",
    data: formattedSignals,
  })
)
  .toChatMessages()
  .at(-1);

Error Message and Stack Trace (if applicable)

BadRequestError: 400 Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.
    at APIError.generate (/Users/nikhilshinday/buildbetter-app/node_modules/openai/error.js:45:20)
    at OpenAI.makeStatusError (/Users/nikhilshinday/buildbetter-app/node_modules/openai/core.js:293:33)
    at OpenAI.makeRequest (/Users/nikhilshinday/buildbetter-app/node_modules/openai/core.js:337:30)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /Users/nikhilshinday/buildbetter-app/node_modules/@langchain/openai/dist/chat_models.cjs:1481:29
    at async RetryOperation._fn (/Users/nikhilshinday/buildbetter-app/node_modules/p-retry/index.js:50:12) {
  status: 400,
  headers: {...},
  request_id: 'req_0c79c8c772cfabeba89e97bb98d75f02',
  error: {
    message: "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.",
    type: 'invalid_request_error',
    param: 'max_tokens',
    code: 'unsupported_parameter'
  },
  code: 'unsupported_parameter',
  param: 'max_tokens',
  type: 'invalid_request_error',
  attemptNumber: 1,
  retriesLeft: 6,
  pregelTaskId: 'f4e45ff0-fad2-5649-b6b6-d01e8ca1aa31'
}

Description

The ChatOpenAI model does not support o3-mini when pulled from langchain hub because the model spec is sending parameters that are not supported by the stated model.

System Info

❯ npm info langchain
npm warn Ignoring workspaces for specified package(s)

langchain@0.3.15 | MIT | deps: 12 | versions: 310
Typescript bindings for langchain
https://github.com/langchain-ai/langchainjs/tree/main/langchain/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/langchain/-/langchain-0.3.15.tgz
.shasum: c6c8b17bf20c870795e31515f48abde7d13ccc9d
.integrity: sha512-+DQ4I2iy4b5sErkxo6jAkgmumvhgqLwLB2fmiGl3yDt8+VVZdB1MUULZMzf+6ubarNc7Mwn/sxHUqK4GhEndhg==
.unpackedSize: 2.9 MB

dependencies:
@langchain/openai: >=0.1.0 <0.5.0        openapi-types: ^12.1.3
@langchain/textsplitters: >=0.0.0 <0.2.0 p-retry: 4
js-tiktoken: ^1.0.12                     uuid: ^10.0.0
js-yaml: ^4.1.0                          yaml: ^2.2.1
jsonpointer: ^5.0.1                      zod-to-json-schema: ^3.22.3
langsmith: >=0.2.8 <0.4.0                zod: ^3.22.4

maintainers:
- nfcampos <nuno@boringbits.io>
- jacoblee93 <jacoblee93@gmail.com>
- andrewnguonly <andrewnguonly@gmail.com>
- davidduong <david@duong.cz>
- hwchase17 <hw.chase.17@gmail.com>
- basproul <braceasproul@gmail.com>

dist-tags:
latest: 0.3.15                             tag-for-publishing-older-releases: 0.2.20
next: 0.3.2-rc.0

published a week ago by jacoblee93 <jacoblee93@gmail.com>

❯ npm info @langchain/openai
npm warn Ignoring workspaces for specified package(s)

@langchain/openai@0.4.2 | MIT | deps: 4 | versions: 79
OpenAI integrations for LangChain.js
https://github.com/langchain-ai/langchainjs/tree/main/libs/langchain-openai/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/@langchain/openai/-/openai-0.4.2.tgz
.shasum: 1259bf56c4948ed2301d366e2fe945c29dfb53bc
.integrity: sha512-Cuj7qbVcycALTP0aqZuPpEc7As8cwiGaU21MhXRyZFs+dnWxKYxZ1Q1z4kcx6cYkq/I+CNwwmk+sP+YruU73Aw==
.unpackedSize: 347.7 kB

dependencies:
js-tiktoken: ^1.0.12        openai: ^4.77.0             zod-to-json-schema: ^3.22.3 zod: ^3.22.4

maintainers:
- hwchase17 <hw.chase.17@gmail.com>
- jacoblee93 <jacoblee93@gmail.com>
- basproul <braceasproul@gmail.com>
- eric_langchain <eric@langchain.dev>
- andrewnguonly <andrewnguonly@gmail.com>
- nfcampos <nuno@boringbits.io>
- vbarda <vadym@langchain.dev>
- davidduong <david@duong.cz>
- maddyadams <madelyn.adams1@gmail.com>
- sam_noyes <sam@langchain.dev>
- benjamincburns <benjamin.c.burns@gmail.com>

dist-tags:
latest: 0.4.2     next: 0.4.0-rc.0

published a week ago by jacoblee93 <jacoblee93@gmail.com>

[dosubot]

Hey @nikhilshinday! I'm here to help you with any bugs, questions, or contributions you have in mind. Let me know how I can assist you!

The error you're encountering is due to the o3-mini model not supporting the max_tokens parameter. Instead, it requires the use of max_completion_tokens. This is a model-specific requirement, and the error message indicates that the parameter being sent is not compatible with the o3-mini model.

To resolve this issue, you should modify the code to use max_completion_tokens instead of max_tokens when configuring the model parameters. This change should align with the requirements of the o3-mini model and prevent the BadRequestError from occurring. Unfortunately, specific details about the o3-mini model's requirements or limitations regarding parameter usage are not available in the current context, so you may need to refer to the model's documentation or configuration guidelines for further details.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[nikhilshinday]

@dosu there is no way to specify max_completion_tokens in the LC hub interface, the only variable you can specify is max_tokens, and it doesn't seem like it's possible to explicitly not set it.

To resolve this issue, you should modify the code to use max_completion_tokens instead of max_tokens when configuring the model parameters. This change should align with the requirements of the o3-mini model and prevent the BadRequestError from occurring. Unfortunately, specific details about the o3-mini model's requirements or limitations regarding parameter usage are not available in the current context, so you may need to refer to the model's documentation or configuration guidelines for further details.

[dosubot]

To configure the LC hub interface to use max_completion_tokens instead of max_tokens for the o3-mini model, you should set the maxCompletionTokens parameter when creating an instance of the ChatCerebras class. This parameter is used internally to set max_completion_tokens in the invocation parameters.

Here's an example of how you can instantiate the ChatCerebras class with the maxCompletionTokens parameter:

import { ChatCerebras } from '@langchain/cerebras';

const llm = new ChatCerebras({
  model: "o3-mini",
  maxCompletionTokens: 100, // Set your desired max completion tokens here
  temperature: 0.7,
  // other params...
});

This will configure the model to use max_completion_tokens as specified ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[jacoblee93]

Oof, thanks for flagging - will coordinate with @madams0013 and fix.

[nikhilshinday]

@jacoblee93 and @madams0013, just following up here - assuming that this won't be updated in the SDK/platform for a while, is there a workaround for the meantime?

[jacoblee93]

Hey sorry for losing track of this, will dig in today!

[jacoblee93]

Just pushed @langchain/openai@0.4.4 that should fix this - can you bump your package version?