Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENH: Assign worker sub pool at runtime instead of pre-allocated #437

Merged
merged 19 commits into from
Sep 13, 2023
Merged

ENH: Assign worker sub pool at runtime instead of pre-allocated #437

merged 19 commits into from
Sep 13, 2023

Conversation

ChengjieLi28
Copy link
Contributor

No description provided.

@XprobeBot XprobeBot added the enhancement New feature or request label Sep 7, 2023
@XprobeBot XprobeBot added this to the v0.4.0 milestone Sep 7, 2023
@ChengjieLi28 ChengjieLi28 marked this pull request as ready for review September 8, 2023 09:16
aresnow1
aresnow1 reviewed Sep 8, 2023
View reviewed changes
xinference/core/worker.py Outdated Show resolved Hide resolved
xinference/client.py Outdated Show resolved Hide resolved
UranusSeven
UranusSeven reviewed Sep 11, 2023
View reviewed changes
xinference/client.py Outdated Show resolved Hide resolved
xinference/client.py Outdated Show resolved Hide resolved
xinference/core/supervisor.py Show resolved Hide resolved
xinference/model/llm/pytorch/core.py Outdated Show resolved Hide resolved
xinference/core/worker.py Show resolved Hide resolved
xinference/core/worker.py Show resolved Hide resolved
xinference/core/worker.py Outdated Show resolved Hide resolved
xinference/core/worker.py Show resolved Hide resolved
xinference/core/worker.py Show resolved Hide resolved
xinference/model/llm/pytorch/core.py Outdated Show resolved Hide resolved
ChengjieLi28 and others added 2 commits September 11, 2023 16:46
@ChengjieLi28
fix some comments
0477f1d
@ChengjieLi28 @UranusSeven
Apply suggestions from code review
2c442b4 
Co-authored-by: Uranus <109661872+UranusSeven@users.noreply.github.com>
@aresnow1
Copy link
Contributor

We should support n_gpu argument when launch model using command lines like this: xinference launch -n llama-2-chat -f ggmlv3 --g-gpu 2.

@XprobeBot XprobeBot modified the milestones: v0.4.0, v0.4.2 Sep 12, 2023
@UranusSeven
Copy link
Contributor

Better rebase the PR to include the CI fix.

@UranusSeven UranusSeven merged commit 14364aa into xorbitsai:main Sep 13, 2023
@UranusSeven UranusSeven mentioned this pull request Sep 25, 2023
Closed
@UranusSeven UranusSeven mentioned this pull request Nov 3, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aresnow1 aresnow1 aresnow1 left review comments

@UranusSeven UranusSeven UranusSeven approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
v0.4.2
Development

Successfully merging this pull request may close these issues.
4 participants
@ChengjieLi28 @aresnow1 @UranusSeven @XprobeBot