The Neighbor Exchange Algorithm for Allgather #822
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Can one of the admins verify this patch? |
|
ok to test |
ikryukov
force-pushed
the
allgather_neighbor_exchange
branch
2 times, most recently
from
2a24add to
9d9c3da
Compare
|
bot:retest |
ikryukov
force-pushed
the
allgather_neighbor_exchange
branch
from
6c33bb3 to
18669f0
Compare
shimmybalsam
approved these changes
Aug 27, 2023
|
@Sergei-Lebedev should we add here ranks reordering? I assume it would improve performance substantially since each neighbor with default mapping will always cross cores... |
this algorithm won't work with reordering, if need reordering we should fallback to ring |
samnordmann
reviewed
Aug 28, 2023
ikryukov
force-pushed
the
allgather_neighbor_exchange
branch
2 times, most recently
from
d7034c7 to
22040c3
Compare
samnordmann
reviewed
Sep 3, 2023
There was a problem hiding this comment.
Thanks @ikryukov for the revisions! It looks good to me. Can you just fix the last comment please?
samnordmann
approved these changes
Sep 4, 2023
Sergei-Lebedev
approved these changes
Sep 4, 2023
ikryukov
force-pushed
the
allgather_neighbor_exchange
branch
2 times, most recently
from
226348e to
369b138
Compare
initial implementation
ikryukov
force-pushed
the
allgather_neighbor_exchange
branch
from
369b138 to
7b9acf2
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Implementation of Neighbor Exchange Allgather algorithm described in this paper: https://ieeexplore.ieee.org/document/1592302
Based on original implementation from OpenMPI: https://github.com/open-mpi/ompi/blob/8514e717215a42ddb6db065c3baae3a2779a072a/ompi/mca/coll/base/coll_base_allgather.c#L624
Why ?
According to mentioned paper:
For the TCP/IP protocol, an algorithm that incorporates pair-wise exchange is the best for minimizing TCP/IP trafficand this algorithm require N / 2 steps instead of N - 1 like in ring.