We provide detailed analysis and code for comparing RMoE with other MoE methods in ./analysis folder.
The ./llama_moe_rnn folder contains the implementation of RMoE in PyTorch based on the llama framework with transformers.
-
Notifications
You must be signed in to change notification settings - Fork 0
qiuzh20/RMoE
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Official implementation of RMoE (Layerwise Recurrent Router for Mixture-of-Experts)
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published