Attention Sinks #1352
lukasfolle
started this conversation in
Ideas
Attention Sinks
#1352
Replies: 1 comment
-
It will somehow help—at least by allowing an infinite output length for the chat model, which could potentially assist with issue #1349. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Not 100% sure if this is still SOTA but do you plan to bring the idea of attention sinks into Tabby?
https://github.com/tomaarsen/attention_sinks?tab=readme-ov-file
I think the idea is really awesome and could help dealing with especially large repos - even combined with treesitter.
Beta Was this translation helpful? Give feedback.
All reactions