Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

anthropic[patch]: Allow 8K maxtokens header for Anthropic while using tools #6480

Merged
merged 7 commits into from
Aug 10, 2024

Conversation

renebrandel
Copy link
Contributor

@renebrandel renebrandel commented Aug 9, 2024

Fixes # (issue)

Right now, you can't use tools and Anthropic's 8K maxTokens feature. I root-caused the issue to langchain always setting the "tools-2024-04-04" header, when tools are used. As far as I know, this tools beta header is deprecated and not in use anymore. This change allows customers to set the maxTokens feature and use tools with Anthropic.

This is aligned with the Python implementation of Langchain, which doesn't conditionally overwrite the headers based on the availability of tools. https://github.com/langchain-ai/langchain/blob/master/libs/partners/anthropic/langchain_anthropic/chat_models.py

@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Aug 9, 2024
Copy link

vercel bot commented Aug 9, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 9, 2024 9:53pm
langchainjs-docs ✅ Ready (Inspect) Visit Preview Aug 9, 2024 9:53pm

@renebrandel renebrandel changed the title Removed unnecessary header for tools to allow for maxtokens header Allow 8K maxtokens header for Anthropic while using tools Aug 9, 2024
@dosubot dosubot bot added the auto:bug Related to a bug, vulnerability, unexpected error with an existing feature label Aug 9, 2024
@jacoblee93 jacoblee93 changed the title Allow 8K maxtokens header for Anthropic while using tools anthropic[patch]: Allow 8K maxtokens header for Anthropic while using tools Aug 9, 2024
@jacoblee93
Copy link
Collaborator

CC @bracesproul but this looks good to me

@renebrandel
Copy link
Contributor Author

@jacoblee93 - I fixed the build error. Silly oversight on my end earlier. Sorry for the churn.

@jacoblee93
Copy link
Collaborator

No worries thank you!

@jacoblee93
Copy link
Collaborator

Can also land ourselves if CI is burdensome

@renebrandel
Copy link
Contributor Author

renebrandel commented Aug 9, 2024

I've just run through all tests locally now and all are green. So I think this build is good to go in CI.

@renebrandel
Copy link
Contributor Author

@jacoblee93 looks like the docs and api reference checks have passed. LMK if there's any more edits needed on my end to kick off the CI.

@bracesproul
Copy link
Member

@renebrandel just kicked off CI again, hold off on updating the branch with main for now since that'll require us to run CI again.

@renebrandel
Copy link
Contributor Author

Sounds good thanks for the update and context.

@renebrandel
Copy link
Contributor Author

@bracesproul looks like it's ready to merge? Anything else I can assist with? Changelog authoring etc?

@jacoblee93 jacoblee93 merged commit 2929159 into langchain-ai:main Aug 10, 2024
34 checks passed
@jacoblee93
Copy link
Collaborator

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature size:S This PR changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants