Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mscclpp-test] Add AllToAll tests #87

Merged
merged 4 commits into from
Jun 6, 2023
Merged

Conversation

Binyang2014
Copy link
Contributor

@Binyang2014 Binyang2014 commented May 29, 2023

For 1n-8p

#
# Initializing MSCCL++
# Setting up the connection in MSCCL++
#
#                                    in-place                       out-of-place          
#       size         count     time   algbw   busbw  #wrong     time   algbw   busbw  #wrong
#        (B)    (elements)     (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
  1073741824      33554432                                    7186.2  149.42  130.74      0

# Out of bounds values : 0 OK
#

For 2n-8p

[1,0]<stdout>:# Initializing MSCCL++
[1,0]<stdout>:# Setting up the connection in MSCCL++
[1,0]<stdout>:#
[1,0]<stdout>:#                                    in-place                       out-of-place          
[1,0]<stdout>:#       size         count     time   algbw   busbw  #wrong     time   algbw   busbw  #wrong
[1,0]<stdout>:#        (B)    (elements)     (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
[1,0]<stdout>:  1073741824      16777216                                     23275   46.13   43.25      0
[1,0]<stdout>:
[1,0]<stdout>:# Out of bounds values : 0 OK

For kernel 1
1n-8p

[1,0]<stdout>:# Initializing MSCCL++
[1,0]<stdout>:# Setting up the connection in MSCCL++
[1,0]<stdout>:#
[1,0]<stdout>:#                                    in-place                       out-of-place          
[1,0]<stdout>:#       size         count     time   algbw   busbw  #wrong     time   algbw   busbw  #wrong
[1,0]<stdout>:#        (B)    (elements)     (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
[1,0]<stdout>:  1073741824      33554432                                    4111.8  261.14  228.50      0
[1,0]<stdout>:
[1,0]<stdout>:# Out of bounds values : 0 OK
[1,0]<stdout>:#

@Binyang2014 Binyang2014 marked this pull request as ready for review June 1, 2023 03:07
Copy link
Contributor

@chhwang chhwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@chhwang chhwang changed the title A naive implementation of alltoall [mscclpp-test] Add AllToAll tests Jun 1, 2023
@chhwang chhwang merged commit bb41c03 into binyli/all-reduce Jun 6, 2023
@chhwang chhwang deleted the binyli/all-to-all branch June 6, 2023 10:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants