[Sonic MoE] Support Sonic MoE Router #297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Draft

hushenwei2000 wants to merge 1 commit into PaddlePaddle:develop from hushenwei2000:sonic_moe

Collaborator

hushenwei2000 commented Jan 5, 2026

需要拉取 hushenwei2000/sonic-moe@5e80341 更改了 Sonic MoE 源码的：

先进行激活函数（softmax/sigmoid），再进行 TopK
支持 sigmoid 激活函数

目前仅测试了 GLM4.5 标准测试。测试结果


          support sonic moe router

f10f1de

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet