Reset kv cache after each query and infinite inference features#2560

Open

pseelam02 wants to merge 4 commits intoggml-org:masterfrom

pseelam02:Reset_and_infinite_inference

Commits on Nov 12, 2024

Added infinite conversation feature to talk-llama
Pranav Aditya Seelam
committed

Commits on Nov 14, 2024

Added reset kv cache after question feature and adjust kv cache when space is full
Pranav Aditya Seelam
committed
Fixed inf loop
Pranav Aditya Seelam
committed
Merge branch 'ggerganov:master' into Reset_and_infinite_inference
pseelam02
authored