Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stops talking mid sentence #988

Closed
Asory2010 opened this issue Apr 14, 2023 · 2 comments
Closed

Stops talking mid sentence #988

Asory2010 opened this issue Apr 14, 2023 · 2 comments

Comments

@Asory2010
Copy link

I am using the vicuna model with the DAN prompt, it works well but it sometimes stops mid sentence for example it would say: "example senten" then stop talking why does this happen?
here is my run.bat script:

title Vicuna!(;
:start
main -i -f prompts/DAN.txt --memory_f32 --interactive-first -r "### Human:" --temp 0 -c 2048 -n -1 --ignore-eos --repeat_penalty 1.2 --instruct --color -m ggml-vicuna-7b-4bit.bin -n 128
pause
goto start

@rabidcopy
Copy link
Contributor

The second -n 128 is the reason for that. Every 128 tokens it's going to stop. Set -n -1 and remove -n 128.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants
@prusnak @rabidcopy @Asory2010 and others