Fix/improve mtp performance#5

Closed

SamuelOliveirads wants to merge 37 commits intoF1LM1:glm4-moe-mtpfrom

SamuelOliveirads:fix/improve-mt-performance

Commits on Aug 11, 2025

Commits on Aug 12, 2025

Commits on Aug 13, 2025

broad thrust of the mtp implementation
F1LM1
committed

Commits on Aug 16, 2025

failed attempt to implement MTP; outputs tokens but KV cache management is unreasonable
F1LM1
committed

Commits on Aug 17, 2025

added proper KV cache management for MTP layers and slightly refactored
F1LM1
committed

Commits on Aug 18, 2025

fixed mtp kv cache update sequencing after prompt processing
F1LM1
committed

Commits on Aug 19, 2025

kludge-y kv cache management of mtp layer
F1LM1
committed

Commits on Aug 20, 2025

fixed vram leak
F1LM1
committed

Commits on Aug 26, 2025

replace standard sampler with greedy sampler for mtp draft
F1LM1
committed

Commits on Sep 2, 2025

fixed mtp kv cache update step in cases where prompt size > n_batch and n_ubatch
F1LM1
committed

Commits on Sep 3, 2025

Commits on Sep 6, 2025

feat: apply logits + greedy sampler
SamuelOliveirads
committed

Commits on Sep 13, 2025

Merge pull request #1 from SamuelOliveirads/glm4-moe-mtp
F1LM1
authored

Commits on Sep 14, 2025

mtp-batch (wip): move mtp execution to batch format
SamuelOliveirads
committed

Commits on Sep 22, 2025

Commits on Sep 24, 2025

mtp-batch (fix): warm mtp cache for small batch size
SamuelOliveirads
committed

Commits on Sep 27, 2025

Commits on Oct 5, 2025

mtp-batch (wip): fix how to warmup kv cache for MTP
SamuelOliveirads
committed

Commits on Oct 9, 2025

mtp-batch (feat): Create and manage sinfo for MTP
SamuelOliveirads
committed

Commits on Oct 10, 2025

Commits on Oct 11, 2025

Commits on Oct 12, 2025

Commits on Oct 16, 2025