-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
anthropic[patch]: Allow 8K maxtokens header for Anthropic while using tools #6480
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
CC @bracesproul but this looks good to me |
@jacoblee93 - I fixed the build error. Silly oversight on my end earlier. Sorry for the churn. |
No worries thank you! |
Can also land ourselves if CI is burdensome |
I've just run through all tests locally now and all are green. So I think this build is good to go in CI. |
@jacoblee93 looks like the docs and api reference checks have passed. LMK if there's any more edits needed on my end to kick off the CI. |
@renebrandel just kicked off CI again, hold off on updating the branch with main for now since that'll require us to run CI again. |
Sounds good thanks for the update and context. |
@bracesproul looks like it's ready to merge? Anything else I can assist with? Changelog authoring etc? |
Thank you! |
Fixes # (issue)
Right now, you can't use tools and Anthropic's 8K maxTokens feature. I root-caused the issue to langchain always setting the "tools-2024-04-04" header, when tools are used. As far as I know, this tools beta header is deprecated and not in use anymore. This change allows customers to set the maxTokens feature and use tools with Anthropic.
This is aligned with the Python implementation of Langchain, which doesn't conditionally overwrite the headers based on the availability of tools. https://github.com/langchain-ai/langchain/blob/master/libs/partners/anthropic/langchain_anthropic/chat_models.py