Skip to content

Conversation

@krissetto
Copy link
Contributor

Calculate max_tokens value for Anthropic requests based on the available context using the official token counting API.

The PR fixes errors such as:

400 Bad Request

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "input length and `max_tokens` exceed context limit: 178959 + 64000 > 200000, decrease input length or `max_tokens` and try again"
  },
  "request_id":"req_011CUB2XUHWkFnQzxcKBbjpF"
}

Closes #569

…context using official token counting api

Signed-off-by: Christopher Petito <chrisjpetito@gmail.com>
@krissetto krissetto requested a review from a team as a code owner October 21, 2025 12:56
@krissetto krissetto added kind/bug Something isn't working area/anthropic labels Oct 21, 2025
@krissetto
Copy link
Contributor Author

Right now I implemented this calculation using the official Anthropic count tokens API.

Another option would be to do some approximated calculation with some external library, and use that estimate instead of an additional API call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/anthropic kind/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic - input tokens + max_tokens > context length

1 participant