You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When loading from a saved --prompt-cache file, llama.cpp appears to initialize response generation with the same state that was saved with the cache-file. This yields the same output every run.
Initial run
% ./main --model ../../Models/LLaMA/7B/ggml-model-q5_1.bin --prompt-cache test-cachefile -n 36 --color -p 'What happens with a test prompt?'
main: build = 537 (6456a4e)
main: seed = 1684011989
[snip]
main: attempting to load saved session from 'test-cachefile'
main: session file does not exist, will create
[snip] What happens with a test prompt? A test prompt is an assignment that helps you find out whether something works.
This can be anything from finding out if your code does what it should do or if the data
next run
% ./main --model ../../Models/LLaMA/7B/ggml-model-q5_1.bin --prompt-cache test-cachefile -n 36 --color -p 'What happens with a test prompt?'
main: build = 537 (6456a4e)
main: seed = 1684012061
[snip]
main: attempting to load saved session from 'test-cachefile'
main: loaded a session with prompt size of 8 tokens
main: session file has exact match for prompt!
[snip] What happens with a test prompt? A test prompt is an assignment that helps you find out whether something works.
This can be anything from finding out if your code does what it should do or if the data
another run, but with explicit --seed
% ./main --model ../../Models/LLaMA/7B/ggml-model-q5_1.bin --prompt-cache test-cachefile -n 36 --color -p 'What happens with a test prompt?' --seed 1234
main: build = 537 (6456a4e)
main: seed = 1234
[snip]
main: attempting to load saved session from 'test-cachefile'
main: loaded a session with prompt size of 8 tokens
main: session file has exact match for prompt!
[snip] What happens with a test prompt? A test prompt is an assignment that helps you find out whether something works.
This can be anything from finding out if your code does what it should do or if the data
Desired Behavior
If loading from a cache-file AND "--seed" # is given, llama.cpp should initialize the random number generator before response generation, so that different responses are generated.
If loading from a cache-file and "--seed #" not given, preserve current behavior.
The text was updated successfully, but these errors were encountered:
Current Behavior
When loading from a saved --prompt-cache file, llama.cpp appears to initialize response generation with the same state that was saved with the cache-file. This yields the same output every run.
Initial run
% ./main --model ../../Models/LLaMA/7B/ggml-model-q5_1.bin --prompt-cache test-cachefile -n 36 --color -p 'What happens with a test prompt?'
main: build = 537 (6456a4e)
main: seed = 1684011989
[snip]
main: attempting to load saved session from 'test-cachefile'
main: session file does not exist, will create
[snip]
What happens with a test prompt?
A test prompt is an assignment that helps you find out whether something works.
This can be anything from finding out if your code does what it should do or if the data
next run
% ./main --model ../../Models/LLaMA/7B/ggml-model-q5_1.bin --prompt-cache test-cachefile -n 36 --color -p 'What happens with a test prompt?'
main: build = 537 (6456a4e)
main: seed = 1684012061
[snip]
main: attempting to load saved session from 'test-cachefile'
main: loaded a session with prompt size of 8 tokens
main: session file has exact match for prompt!
[snip]
What happens with a test prompt?
A test prompt is an assignment that helps you find out whether something works.
This can be anything from finding out if your code does what it should do or if the data
another run, but with explicit --seed
% ./main --model ../../Models/LLaMA/7B/ggml-model-q5_1.bin --prompt-cache test-cachefile -n 36 --color -p 'What happens with a test prompt?' --seed 1234
main: build = 537 (6456a4e)
main: seed = 1234
[snip]
main: attempting to load saved session from 'test-cachefile'
main: loaded a session with prompt size of 8 tokens
main: session file has exact match for prompt!
[snip]
What happens with a test prompt?
A test prompt is an assignment that helps you find out whether something works.
This can be anything from finding out if your code does what it should do or if the data
Desired Behavior
If loading from a cache-file AND "--seed" # is given, llama.cpp should initialize the random number generator before response generation, so that different responses are generated.
If loading from a cache-file and "--seed #" not given, preserve current behavior.
The text was updated successfully, but these errors were encountered: