Skip to content

[Fix] Fall back to Gloo when NCCL backend is unavailable #19641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 17, 2025
Merged

[Fix] Fall back to Gloo when NCCL backend is unavailable #19641

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions vllm/distributed/parallel_state.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -938,6 +938,13 @@ def init_distributed_environment(
assert distributed_init_method is not None, (
"distributed_init_method must be provided when initializing "
"distributed environment")
if not torch.distributed.is_backend_available(backend):
logger.warning(
"Distributed backend %s is not available; "
"falling back to gloo.", backend)
assert torch.distributed.is_gloo_available(), (
"Fallback Gloo backend is not available.")
backend = "gloo"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to check "gloo" is available in torch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gloo is kind of the most basic distributed backend we have, right? We can check if Gloo is available before trying to fall back to it, but if it's also not available, all we can do is throw an error and exit, which is exactly what will already happen if we just try to use gloo anyway.

Happy to put in a check if you think it's worth it for the better error message, though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just assert torch.distributed.is_gloo_available().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

# this backend is used for WORLD
torch.distributed.init_process_group(
backend=backend,
Expand Down