Dev2049/openllm #6578
Merged
Dev2049/openllm #6578
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!-- Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution. Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change. After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost. Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle! --> <!-- Remove if not applicable --> 🎉 OpenLLM 🤝 LangChain [OpenLLM](https://github.com/bentoml/OpenLLM) is a new open platform for operating large language models(LLMs) in production. Serve, deploy, and monitor any LLMs with ease. OpenLLM lets developers and researchers to easily run inference with any open-source LLMs, deploy to the cloud or on-premises, build powerful AI apps, and fine tune your own LLM (coming soon...) It currently supports ChatGLM, Dolly-v2, Flan-T5, Falcon, Starcoder, and more to come. One can also easily start either a REST/gRPC server, which is powered by BentoML. Now that's out of the way, lets dive in! The current depenedencies for this integration: `openllm` This integrations brings a `OpenLLM` llms to LangChain, that can be used for both running LLMs locally as well as interacting with a remote OpenLLM server. To quickly start a local LLM, simply do the following: ```python from langchain.llms import OpenLLM llm = OpenLLM(model_name="dolly-v2", model_id='databricks/dolly-v2-7b', device_map='auto') llm("What is the difference between a duck and a goose? And why there are so many Goose in Canada?") ``` `langchain.llms.OpenLLM`, as mentioned above also have the capabilities to interact with remote OpenLLM Server. Given there is a running OpenLLM server at http://44.23.123.1, you can do the following: ```python from langchain.llms import OpenLLM llm = OpenLLM(server_url='http://44.23.123.1:3000', server_type='grpc') llm("What is the difference between a duck and a goose? And why there are so many Goose in Canada?") ``` Features soon to be open-sourced: - OpenAI compatible API, allowing users to easily use LangChain's OpenAI llm. - SSE support for OpenLLM server, allowing users to stream inference results. - Last but not least, easily fine-tune your own LLMs with `LLM.tuning()` Last but not least, I would love to hear feedback and response from the community about the project, and feel free to reach out to me via Twitter @aarnphm_. Feel free to join our [Discord](https://l.bentoml.com/join-openllm-discord) to get the latest updates and developments. Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com> #### Before submitting <!-- If you're adding a new integration, please include: 1. a test for the integration - favor unit tests that does not rely on network access. 2. an example notebook showing its use See contribution guidelines for more information on how to write tests, lint etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md --> I have added tests for this integration. #### Who can review? Tag maintainers/contributors who might be interested: <!-- For a quicker response, figure out the right person to tag with @ @hwchase17 - project lead Tracing / Callbacks - @agola11 Async - @agola11 DataLoaders - @eyurtsev Models - @hwchase17 - @agola11 Agents / Tools / Toolkits - @hwchase17 VectorStores / Retrievers / Memory - @dev2049 --> cc @hwchase17 @agola11 --------- Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com> Co-authored-by: Chaoyu <paranoyang@gmail.com> Co-authored-by: Dev 2049 <dev.dev2049@gmail.com>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
Merged
tconkling
added a commit
to tconkling/langchain
that referenced
this pull request
Jun 22, 2023
* master: MD header text splitter returns Documents (langchain-ai#6571) Fix callback forwarding in async plan method for OpenAI function agent (langchain-ai#6584) bump 209 (langchain-ai#6593) Clarifai integration (langchain-ai#5954) Add missing word in comment (langchain-ai#6587) Add AzureML endpoint LLM wrapper (langchain-ai#6580) Add OpenLLM wrapper(langchain-ai#6578) feat: interfaces for async embeddings, implement async openai (langchain-ai#6563) Upgrade the version of AwaDB and add some new interfaces (langchain-ai#6565) add motherduck docs (langchain-ai#6572) Detailed using the Twilio tool to send messages with 3rd party apps incl. WhatsApp (langchain-ai#6562) Change Data Loader Namespace (langchain-ai#6568) Remove duplicate databricks entries in ecosystem integrations (langchain-ai#6569) Fix whatsappchatloader - enable parsing new datetime format on WhatsApp chat (langchain-ai#6555) Wait for all futures (langchain-ai#6554) feat: faiss filter from list (langchain-ai#6537) update pr tmpl (langchain-ai#6552) Remove unintended double negation in docstring (langchain-ai#6541) Minor Grammar Fixes in Docs and Comments (langchain-ai#6536)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
update poetry lock