adding support for Palm, Cohere, HF Inference Endpoints, Replicate #349
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Can you showcase how would we actually use Palm, Cohere, HF Inference Endpoints? Thanks |
|
Yes! will do |
Update Readme with usage instructions
|
Updated ReadME Also tested this on local with AI21 - https://www.ai21.com/studio/pricing 
|
|
In addition, we also enable easy A/B testing of new LLM models in production: https://docs.litellm.ai/docs/tutorials/ab_test_llms Happy to add additional tests / update documentation, if the initial PR looks good to you. |
|
Bump on this @pycui @Shaunwei @ZongZiWang |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi @pycui @Shaunwei @ZongZiWang
Noticed you're calling Anthropic and OpenAI. I'm working on litellm (simple library to standardize LLM API Calls - https://github.com/BerriAI/litellm) and was wondering if we could be helpful.
I added support for palm, cohere, replicate, and huggingface inference endpoints by replacing the ChatOpenAI langchain integration with ChatLiteLLM. The code is pretty similar to the OpenAI class - as litellm follows the same pattern as the openai-python sdk.
Would love to know if this helps.
Happy to add additional tests / update documentation, if the initial PR looks good to you.