Replies: 1 comment
-
Yes, change the WI insert location to "Before A/N" instead of "After Memory", and the amount to reprocess should be greatly reduced. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I use Context Shifting, and my context size (4 example) 32k. If I use world info and set it to only consider the last 4096 context, when inserting the World Info keys, it will insert after the memory / AN without any issues and extra reprocessing. However when the mentioned token is further away than 4096, it will reload all of 32K tokens to clear out the World Info keys from the context buffer. Is there any way to avoid this reprocessing?
Thank you in advance! Koboldcpp is the best! ^^
Beta Was this translation helpful? Give feedback.
All reactions