Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc]: Update the vllm distributed Inference and Serving with the new MultiprocessingGPUExecutor #5221

Closed
rcarrata opened this issue Jun 3, 2024 · 3 comments · Fixed by #5230
Closed

[Doc]: Update the vllm distributed Inference and Serving with the new MultiprocessingGPUExecutor #5221

rcarrata opened this issue Jun 3, 2024 · 3 comments · Fixed by #5230
Labels
documentation Improvements or additions to documentation

Comments

@rcarrata
Copy link

rcarrata commented Jun 3, 2024

📚 The doc issue

The vLLM documentation only reflects the possibility to use Ray for running Distributed Inference and Serving with vLLM, even though the #4539 issue is merged and v0.4.3 is released with the MultiprocessingGPUExecutor feature included as an alternative to Ray for single-node inferencing.

Suggest a potential alternative/fix

Update the documentation to reflect the possibility of using MultiprocessingGPUExecutor as an alternative to Ray for single-node inferencing.
@rcarrata rcarrata added the documentation Improvements or additions to documentation label Jun 3, 2024
@youkaichao
Copy link
Member

Contributions are welcome!

njhill added a commit to njhill/vllm that referenced this issue Jun 3, 2024
@njhill
[Core][Doc] Default to multiprocessing for single-node distributed case
89fe0f5 
Also update docs to reflect support for the multiprocessing distributed executor.

Resolves vllm-project#4955
Resolves vllm-project#5221
@njhill njhill mentioned this issue Jun 3, 2024
Merged
@njhill
Copy link
Member

njhill commented Jun 3, 2024

I have included these doc updates in #5230.

@rcarrata
Copy link
Author

rcarrata commented Jun 4, 2024

thanks for your help @njhill, much appreciated!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Core][Doc] Default to multiprocessing for single-node distributed case njhill/vllm
3 participants
@rcarrata @njhill @youkaichao