Skip to content

Commit

Permalink
update readme (#14)
Browse files Browse the repository at this point in the history 
* update readme

* catch up to latest nccl executor
  • Loading branch information
@Andyli1007
Andyli1007 authored Nov 6, 2023
1 parent a1ea0f9 commit 5d2599b
Show file tree
Hide file tree
Showing 4 changed files with 24 additions and 190 deletions.
1 change: 1 addition & 0 deletions .gitmodules
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@
[submodule "scheduler/msccl-scheduler"]
path = scheduler/msccl-scheduler
url = https://github.com/Azure/msccl-scheduler
branch = main
117 changes: 21 additions & 96 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,228 +15,139 @@ MSCCL vision is to provide a unified, efficient, and scalable framework for exec
- MSCCL test toolkit([msccl-tests-nccl](https://github.com/Azure/msccl-tests-nccl)): These tests check both the performance and the correctness of MSCCL operations.

## Performance
For reference, FP16 All-Reduce and All-Gather algorithms were tested and compared on ND H100 v5 VM, using msccl-tests-nccl.
For reference, FP16 All-Gather algorithms were tested and compared on ND H100 v5 VM, using msccl-tests-nccl.

<table>
<tr>
<th colspan="4">FP16 All-Reduce Latency (us)</th>
<th colspan="4">All-Gather Latency (us)</th>
</tr>
<tr>
<th>Message Size</th>
<th>NCCL</th>
<th>MSCCL</th>
<th>MSCCL Speedup</th>
<th>Message Size</th>
<th>NCCL</th>
<th>MSCCL</th>
<th>MSCCL Speedup</th>
</tr>
<tr>
<td>1KB</td>
<td>13.12</td>
<td>5.84</td>
<td>2.25x</td>
<td>1KB</td>
<td>9.54</td>
<td>5.65</td>
<td>1.69x</td>
</tr>
<tr>
<td>2KB</td>
<td>14.39</td>
<td>5.9</td>
<td>2.44x</td>
<td>2KB</td>
<td>9.8</td>
<td>5.7</td>
<td>1.72x</td>
</tr>
<tr>
<td>4KB</td>
<td>15.28</td>
<td>5.83</td>
<td>2.62x</td>
<td>4KB</td>
<td>9.78</td>
<td>5.43</td>
<td>1.80x</td>
</tr>
<tr>
<td>8KB</td>
<td>15.69</td>
<td>5.87</td>
<td>2.67x</td>
<td>8KB</td>
<td>9.78</td>
<td>5.47</td>
<td>1.81x</td>
</tr>
<tr>
<td>16KB</td>
<td>16.64</td>
<td>5.94</td>
<td>2.80x</td>
<td>16KB</td>
<td>10.29</td>
<td>5.53</td>
<td>1.86x</td>
</tr>
<tr>
<td>32KB</td>
<td>19.3</td>
<td>6.14</td>
<td>3.14x</td>
<td>32KB</td>
<td>12.49</td>
<td>5.75</td>
<td>2.17x</td>
</tr>
<tr>
<td>64KB</td>
<td>20</td>
<td>6.47</td>
<td>3.09x</td>
<td>64KB</td>
<td>12.87</td>
<td>5.95</td>
<td>2.16x</td>
</tr>
<tr>
<td>128KB</td>
<td>20.42</td>
<td>7.57</td>
<td>2.70x</td>
<td>128KB</td>
<td>13.16</td>
<td>6.38</td>
<td>2.06x</td>
</tr>
<tr>
<td>256KB</td>
<td>20.5</td>
<td>9.39</td>
<td>2.18x</td>
<td>256KB</td>
<td>13.23</td>
<td>7.26</td>
<td>1.82x</td>
</tr>
<tr>
<td>512KB</td>
<td>29.89</td>
<td>12.58</td>
<td>2.38x</td>
<td>512KB</td>
<td>13.39</td>
<td>8.71</td>
<td>1.54x</td>
</tr>
<tr>
<td>1MB</td>
<td>31.94</td>
<td>18.21</td>
<td>1.75x</td>
<td>1MB</td>
<td>18.33</td>
<td>12.3</td>
<td>1.49x</td>
</tr>
<tr>
<td>2MB</td>
<td>37.95</td>
<td>24.47</td>
<td>1.55x</td>
<td>2MB</td>
<td>23.18</td>
<td>17.75</td>
<td>1.31x</td>
</tr>
<tr>
<td>4MB</td>
<td>49.28</td>
<td>38.23</td>
<td>1.29x</td>
<td>4MB</td>
<td>33.66</td>
<td>23.37</td>
<td>1.44x</td>
</tr>
<tr>
<td>8MB</td>
<td>77.01</td>
<td>74.06</td>
<td>1.04x</td>
<td>8MB</td>
<td>44.7</td>
<td>38.54</td>
<td>1.16x</td>
</tr>
<tr>
<td>16MB</td>
<td>116</td>
<td>115.7</td>
<td>1.00x</td>
<td>16MB</td>
<td>67.19</td>
<td>67.16</td>
<td>1.00x</td>
</tr>
<tr>
<td>32MB</td>
<td>187.2</td>
<td>186.5</td>
<td>1.00x</td>
<td>32MB</td>
<td>104.7</td>
<td>98.4</td>
<td>1.06x</td>
</tr>
<tr>
<td>64MB</td>
<td>317.4</td>
<td>315.7</td>
<td>1.01x</td>
<td>64MB</td>
<td>192.4</td>
<td>181.9</td>
<td>1.06x</td>
</tr>
<tr>
<td>128MB</td>
<td>572.5</td>
<td>570.4</td>
<td>1.00x</td>
<td>128MB</td>
<td>368.3</td>
<td>348.4</td>
<td>1.06x</td>
</tr>
<tr>
<td>256MB</td>
<td>1079</td>
<td>1075.6</td>
<td>1.00x</td>
<td>256MB</td>
<td>699.5</td>
<td>680.7</td>
<td>1.03x</td>
</tr>
<tr>
<td>512MB</td>
<td>2071.1</td>
<td>2067.9</td>
<td>1.00x</td>
<td>512MB</td>
<td>1358.6</td>
<td>1339.3</td>
<td>1.01x</td>
</tr>
<tr>
<td>1GB</td>
<td>4028.7</td>
<td>4026.8</td>
<td>1.00x</td>
<td>1GB</td>
<td>2663.8</td>
<td>2633</td>
Expand All @@ -248,13 +159,13 @@ For reference, FP16 All-Reduce and All-Gather algorithms were tested and compare

In order to use MSCCL, you may follow these steps to use two different MSCCL algorithms for AllReduce on Azure NDv4 which has 8xA100 GPUs:

Follow below steps to download the source code of msccl and related submodules
#####1. Follow below steps to download the source code of msccl and related submodules

```sh
$ git clone https://github.com/Azure/msccl.git --recurse-submodules
```

Steps to install MSCCL executor:
#####2. Steps to install MSCCL executor:

```sh
$ git clone https://github.com/Azure/msccl.git --recurse-submodules
Expand All @@ -264,7 +175,7 @@ $ cd ../
$ cd ../
```

Then, follow these steps to install msccl-tests-nccl for performance evaluation:
#####3. follow these steps to install msccl-tests-nccl for performance evaluation:

```sh
$ cd tests/msccl-tests-nccl/
Expand All @@ -273,7 +184,20 @@ $ cd ../
$ cd ../
```

Next install [MSCCL toolkit](https://github.com/microsoft/msccl-tools) to compile a few custom algorithms:
#####4. apply the msccl algo when using msccl executor
######- for ndv5, we already have algo optimized, you can use msccl scheduler to apply this algo directly to the executor, below is the steps to apply the scheduler
```sh
$ sudo apt-get install libcurl4-openssl-dev nlohmann-json3-dev

for nccl:
$ CXX=/path/to/nvcc BIN_HOME=/path/to/nccl/binary SRC_HOME=/path/to/nccl/source make
for rccl:
$ CXX=/path/to/nvcc BIN_HOME=/path/to/nccl/binary SRC_HOME=/path/to/nccl/source make PLATFORM=RCCL

$ make install
```

######- for customize the msccl algo for your system, you can install [MSCCL toolkit](https://github.com/microsoft/msccl-tools) to compile a few custom algorithms:

```sh
$ git clone https://github.com/microsoft/msccl-tools.git
Expand All @@ -286,11 +210,12 @@ $ cd ../

The compiler's generated code is an XML file (`test.xml`) that is fed to MSCCL runtime. To evaluate its performance, copy the `test.xml` to the msccl/exector/msccl-executor-nccl/build/lib/msccl-algorithms/ and execute the following command line on an Azure NDv4 node or any 8xA100 system:

######below is the command to run test using msccl-executor-nccl
```sh
$ mpirun -np 8 -x LD_LIBRARY_PATH=msccl/exector/msccl-executor-nccl/build/lib/:$LD_LIBRARY_PATH -x NCCL_DEBUG=INFO -x NCCL_DEBUG_SUBSYS=INIT,ENV tests/msccl-tests-nccl/build/all_reduce_perf -b 128 -e 32MB -f 2 -g 1 -c 1 -n 100 -w 100 -G 100 -z 0
```

If everything is installed correctly, you should see the following output in log:
######If everything is installed correctly, you should see the following output in log:

```sh
[0] NCCL INFO Connected 1 MSCCL algorithms
Expand Down
Loading

0 comments on commit 5d2599b

Please sign in to comment.