Skip to content

Tutel v0.2.0
Compare
Choose a tag to compare
@ghostplant ghostplant released this 11 Aug 04:19
· 34 commits to main since this release
v0.2.0
a2242e7

What's New in v0.2.0:

  1. Support Windows Python3 + Torch Installation;
  2. Add examples to enable Tutel MoE in Fairseq;
  3. Refactor MoE Layer implementation, letting all features (e.g. top-X, overlap, parallel_type, capacity, ..) be able to change at different forward interations;
  4. New features: load_importance_loss, cosine router, inequivalent_tokens;
  5. Extend capacity_factor value that includes zero value and negative values for smarter capacity estimation;
  6. Add tutel.checkpoint conversion tools to reformat checkpoint files, making it able to use existing checkpoints to train/infer with a different world size.
How to Setup:
python3 -m pip install --user https://github.com/microsoft/tutel/archive/refs/tags/v0.2.0.tar.gz
Assets 2
Loading