Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Misc][TPU] Support TPU in initialize_ray_cluster #6812

Merged
merged 2 commits into from
Jul 26, 2024
Merged

[Misc][TPU] Support TPU in initialize_ray_cluster #6812

merged 2 commits into from
Jul 26, 2024

Conversation

WoosukKwon
Copy link
Collaborator

This PR adds support for TPUs in initializing Ray cluster.

Will also add HPU in the PR #6143

@WoosukKwon WoosukKwon added the tpu Related to Google TPUs label Jul 26, 2024
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which consists a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of default ones by unblocking the steps in your fast-check build on Buildkite UI.

Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge).

To run full CI, you can do one of these:

  • Comment /ready on the PR
  • Add ready label to the PR
  • Enable auto-merge.

🚀

@WoosukKwon WoosukKwon added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 26, 2024
@WoosukKwon
Copy link
Collaborator Author

This PR was part of #5871, but separated out to get a quick review.

@WoosukKwon WoosukKwon changed the title [Misc] Support TPU in initialize_ray_cluster [Misc][TPU] Support TPU in initialize_ray_cluster Jul 26, 2024
rkooo567
rkooo567 approved these changes Jul 26, 2024
View reviewed changes
Copy link
Collaborator

@rkooo567 rkooo567 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. small nits

@WoosukKwon WoosukKwon enabled auto-merge (squash) July 26, 2024 17:28
@WoosukKwon WoosukKwon merged commit aa48677 into main Jul 26, 2024
73 checks passed
@WoosukKwon WoosukKwon deleted the ray-tpu-gaudi branch July 26, 2024 20:28
cadedaniel pushed a commit to cadedaniel/vllm-public that referenced this pull request Jul 27, 2024
@WoosukKwon @cadedaniel
[Misc][TPU] Support TPU in initialize_ray_cluster (vllm-project#6812)
5474195
@WoosukKwon WoosukKwon mentioned this pull request Jul 27, 2024
Merged
@dtrifiro dtrifiro mentioned this pull request Aug 5, 2024
Closed
kylesayrs pushed a commit to neuralmagic/vllm that referenced this pull request Aug 17, 2024
@WoosukKwon @kylesayrs
[Misc][TPU] Support TPU in initialize_ray_cluster (vllm-project#6812)
0eb98c7
Alvant pushed a commit to compressa-ai/vllm that referenced this pull request Oct 26, 2024
@WoosukKwon @Alvant
[Misc][TPU] Support TPU in initialize_ray_cluster (vllm-project#6812)
054f002 
Signed-off-by: Alvant <alvasian@yandex.ru>
KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024
@WoosukKwon
[Misc][TPU] Support TPU in initialize_ray_cluster (vllm-project#6812)
ffc0b5c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rkooo567 rkooo567 rkooo567 approved these changes

@Yard1 Yard1 Awaiting requested review from Yard1

@cadedaniel cadedaniel Awaiting requested review from cadedaniel

Assignees
No one assigned
Labels
ready ONLY add when PR is ready to merge/full CI is needed tpu Related to Google TPUs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@WoosukKwon @rkooo567