Benchmark results for DeepSeek-v3 in 2x8xH200 Cluster

[roG0d]

Motivation

• Having baseline metrics to compare future works using a 2x8xH200 GPU cluster.
• Explore the tradeoffs of increasing the number of chips with more GPU memory, H200, versus increasing the parallel inference world size when H100.
• Measure multi-node inference overhead compared to single-node.
• Explore the benefits of using FP8 quantization.

For output files and logs, please refer to: https://github.com/datacrunch-research/h200-benchmarks

Modifications

• Added a new folder benchmark_dsv3 resembling a similar structure of other benchmark folders.
• Added deepseek_v3.shscript containing each benchmark performed.
• Added a README.md containing the metrics obtained from the benchmarks performed.

Checklist

• Format your code according to the Contributor Guide.
• Add unit tests as outlined in the Contributor Guide.
• Update documentation as needed, including docstrings or example tutorials.

[zhyncs]

Hi @roG0d Sorry for the late response, may you try the latest version v0.4.1.post4

[roG0d]

Sure, @zhyncs! If you'd like, we were thinking it might be useful to track the progress made in #2591 and benchmark it for single-node FP8 and BF16.

We could create another folder called benchmark_v0_4_1_post to store these benchmark results, similar to what was done for this PR.

[roG0d]

We already have the results for v.0.4.1.post4 up to FusedMoE tuning for H200 here. Would you prefer us to create a new PR for v0.4.1.post4 with these results or should I include it in the current one for main so we can continue updating it with future optimizations?