-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Local llm mutli-gpu support #1391
Conversation
Ubuntu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1391 +/- ##
==========================================
+ Coverage 58.64% 58.67% +0.02%
==========================================
Files 230 230
Lines 11199 11213 +14
Branches 1209 1212 +3
==========================================
+ Hits 6568 6579 +11
- Misses 4292 4294 +2
- Partials 339 340 +1 ☔ View full report in Codecov by Sentry. |
Description
Added multi-gpu support in local llms feature
Command to build without local llms: docker compose -f docker-compose.yaml up --build
Command to build with local llms: docker compose -f docker-compose-gpu.yml up --build
Related Issues
Solution and Design
Test Plan
Type of change
Checklist