fix num_kv_heads sharding in uneven autoTP for Falcon-40b#4712

Merged

mrwyattii merged 4 commits intodeepspeedai:masterfrom

Yejing-Lai:lyj/fix_falcon40b

Jan 5, 2024

Commits on Nov 21, 2023

fix num_kv_heads sharding in uneven autoTP for Falcon-40b
Yejing-Lai
committed

Commits on Dec 13, 2023

Merge branch 'master' into lyj/fix_falcon40b
tjruwase
authored

Commits on Jan 3, 2024

Merge branch 'master' into lyj/fix_falcon40b
tjruwase
authored

Commits on Jan 5, 2024

Merge branch 'master' into lyj/fix_falcon40b
mrwyattii
authored