-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update llm_inference.md #245
Conversation
Signed-off-by: alabulei1 <vivian.xiage@gmail.com>
Hello, I am a PR summary agent on flows.network. Here are my reviews of code commits in this PR. The GitHub patch primarily focuses on updating the documentation for running open-source Large Language Models (LLMs) with WasmEdge and Rust, specifically to reflect that it now supports any open-source LLM model beyond just Llama2. The supported models list is no longer mentioned directly in the documentation but instead, it's stated that WasmEdge can support any open-source LLMs. Potential Issues:
Key Findings:
These changes enhance the application's flexibility by adding support for various LLM models and improving logging capabilities. DetailsCommit 5d00cd4515d6626a1fa44b48fd8cfa18be951f46
Commit 77d94e51ea350abc4ce32f45d639001eb6b35efd
Commit 4e0b7bcaa3ff95b30df7b29e20e3015be41b6e7a
Commit 9bc6f0a042852ecfd6e22a095c9d8635bb74a621
These changes enhance the application's flexibility by adding support for various LLM models and improving logging capabilities. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change the example to llama 3.1.
Signed-off-by: alabulei1 <vivian.xiage@gmail.com>
Signed-off-by: Michael Yuan <michael@secondstate.io>
Explanation
Use a smaller batch size to make sure the
llama-simple.wasm
can work on most machines.Related issue
LlamaEdge/LlamaEdge#199
What type of PR is this