[Distributed] [ROCM] Fix custom allreduce enable checks #16010
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
vllm/config.py
Outdated
There was a problem hiding this comment.
This condition should be inverted?
| if current_platform.use_custom_allreduce(): | |
| if not current_platform.use_custom_allreduce(): |
vllm/config.py
Outdated
There was a problem hiding this comment.
| "Disable the custom all-reduce kernel because it is not " | |
| "supported on on current platform.") | |
| "Disabled the custom all-reduce kernel because it is not " | |
| "supported on current platform.") |
ilmarkov
force-pushed
the
imarkov/fix_ca_enable
branch
from
fa234a2 to
7a2f7c2
Compare
Signed-off-by: ilmarkov <imarkov@redhat.com>
ilmarkov
force-pushed
the
imarkov/fix_ca_enable
branch
from
7a2f7c2 to
f93e47f
Compare
github-actions
bot
added
the
ready
…#16010) Signed-off-by: ilmarkov <imarkov@redhat.com> Co-authored-by: ilmarkov <imarkov@redhat.com> Signed-off-by: xinyuxiao <xinyuxiao2024@gmail.com>
…#16010) Signed-off-by: ilmarkov <imarkov@redhat.com> Co-authored-by: ilmarkov <imarkov@redhat.com> Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>
…#16010) Signed-off-by: ilmarkov <imarkov@redhat.com> Co-authored-by: ilmarkov <imarkov@redhat.com>
…#16010) Signed-off-by: ilmarkov <imarkov@redhat.com> Co-authored-by: ilmarkov <imarkov@redhat.com>
…#16010) Signed-off-by: ilmarkov <imarkov@redhat.com> Co-authored-by: ilmarkov <imarkov@redhat.com> Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
4 participants
As mentioned in the comment to the original PR enabling custom allreduce on ROCM, device capability on RocM is not sequential so the way we enable custom allreduce for MI300 should be changed.
This PR adds
use_custom_allreducemethod to platform class letting enable custom allreduce for ROCM platform ongfx94architectures.