Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Anthropic Claude 3 #443

Closed
enricoros opened this issue Mar 4, 2024 · 10 comments
Closed

Support Anthropic Claude 3 #443

enricoros opened this issue Mar 4, 2024 · 10 comments

Comments

@enricoros
Copy link
Owner

Looking for PR to add support for Claude 3, released today by Anthropic.

@jacksongoode
Copy link
Contributor

#444

@IdkwhatImD0ing
Copy link

This pr does not seem to work for me. Error message stating it requires the new messages api.

@jacksongoode
Copy link
Contributor

Ah there was an API change as well?

@IdkwhatImD0ing
Copy link

It seems like it, other claude models work, just not Claude 3

@Penagwin
Copy link
Contributor

Penagwin commented Mar 5, 2024

https://docs.anthropic.com/claude/docs/models-overview#model-comparison

All Claude 3 models only support the Messages API and I think currently only the completions API is implemented for Claude here?

@enricoros
Copy link
Owner Author

enricoros commented Mar 5, 2024

Correct @Penagwin - they just introduced the new Messages API and the Claude 3 models are the ones to only support that new API. We'll need to update the backend code.

@IdkwhatImD0ing
Copy link

Is this the backend code for claude: https://github.com/enricoros/big-AGI/blob/main/src/modules/llms/server/anthropic/anthropic.router.ts

Might be interested in taking a crack at this

@slapglif
Copy link

slapglif commented Mar 7, 2024

Hey everyone! 👋

I've taken a crack at adding support for Anthropic Claude 3 models, as mentioned by @enricoros and @Penagwin. The changes primarily focus on updating the backend code in anthropic.router.ts to use the new Anthropic Messages API.

Here's a quick rundown of what I've done:

🔧 Updated the backend code to use the new Messages API, as required by Claude 3 models.
🆕 Implemented support for Claude 3 models, so users can now leverage these powerful new tools.
📝 Updated the documentation and comments to reflect the changes and provide clarity on usage.
🧪 Added tests to ensure the Claude 3 functionality works as expected and to maintain code quality.
I've submitted a pull request with all the details, and I'm excited to hear your thoughts and feedback! Let's work together to get this merged and provide our users with access to the latest and greatest from Anthropic.

Thanks for your time, and let me know if you have any questions or suggestions! 😄

#450

@enricoros
Copy link
Owner Author

Thanks @slapglif - looking into it. Seems like there's no Streaming support; looking into that.

@slapglif
Copy link

slapglif commented Mar 7, 2024

@enricoros
no problem, been using this software since the day i hit you up, least i can do - i replied to the pr:

@enricoros as to the fidelity of the fix, upon further inspection, i receive some cases with:

Compiling /api/trpc-node/[trpc] ...
 ✓ Compiled /api/trpc-node/[trpc] in 1037ms (700 modules)
Error in stream: [SyntaxError: Expected ',' or '}' after property value in JSON at position 39]

this is likley a parsing error, ill see to it shortly, as for the lack of streaming support - the current situation is that streaming is supported and streams to console, but i wanted to push a working version before attacking the streaming to next bug,

to address the formatting - sorry yeah python guy, can you define your standard for me so i can use an llm to reformat?

#450 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants