Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix two bugs in kv-cache backtrack loop (mlc-ai#856)
Fix two bugs in kv-cache pop loop Bug 1: old code would stop early because output_ids was shortened in-place during the loop Bug 2: off-by-one in backoff size due to break
- Loading branch information