Skip to content

Tutel v0.1.2
Compare
Choose a tag to compare
@ghostplant ghostplant released this 16 Nov 08:26
· 74 commits to v0.1.x since this release
v0.1.2
6b434d9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

What's New in v0.1.2:

  1. General-purpose top-k gating with {'type': 'top', 'k': 2};
  2. Add Megatron-ML Tensor Parallel as gating type;
  3. Add deepspeed-based & megatron-based helloworld example for fair comparison;
  4. Add torch.bfloat16 datatype support for single-GPU;
How to Setup:
python3 -m pip install --user https://github.com/microsoft/tutel/archive/refs/tags/v0.1.2.tar.gz

Contributors: @ghostplant, @EricWangCN, @foreveronehundred.

Contributors

ghostplant, foreveronehundred, and EricWangCN
Assets 2
Loading