Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using the config alpaca.txt line breaks do not work #969

Closed
Belluxx opened this issue Apr 14, 2023 · 4 comments
Closed

Using the config alpaca.txt line breaks do not work #969

Belluxx opened this issue Apr 14, 2023 · 4 comments
Assignees

Comments

@Belluxx
Copy link

Belluxx commented Apr 14, 2023

I am using a build from the latest f4d277a commit.

This is the specific command that i am running:

./main\
	--model "ggjt-vicuna-q4_0.bin"\
	--mlock\
	--threads 4\
	--seed 69\
	--interactive-first\
	--keep -1\
	--ins-prefix-bos\
	--ins-prefix "\n### Human: "\
	--ins-suffix "\n### Assistant: "\
	--reverse-prompt "### Human: "\
	--rm-trailing-space-workaround\
	--disable-multiline\
	--color\
	--prompt "A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions."

The issue is that i am getting a chat like this:

 A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.\n### Human: Hello
\n### Assistant: Hello! How can I assist you today?
### Human: I would like to have a milkshake please
\n### Assistant: Sure thing! What kind of milkshake would you like? Do you want it to be sweet or savory? Would you like nuts, fruits, or whipped cream on top? Let me know and I'll get started.
### Human: Sweet with fruits please
\n### Assistant: Great! Here's your fruit-filled milkshake:

[Ballpoint Pen Drawing of a Fruit-Filled Milkshake]

![B\n### Human: Thanks
\n### Assistant: You're welcome! Let me know if you have any more questions.

Note all the \n that are printed in plain text instead of becoming line breaks. This happens both in bash and zsh. Also note that where the line break happens is only due to the fact that i pressed enter manually.

The expected behavior is:

 A chat between a curious human and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the human's questions.
### Human: Hello
### Assistant: Hello! How can I assist you today?
### Human: I would like to have a milkshake please
### Assistant: Sure thing! What kind of milkshake would you like? Do you want it to be sweet or savory? Would you like nuts, fruits, or whipped cream on top? Let me know and I'll get started.
### Human: Sweet with fruits please
### Assistant: Great! Here's your fruit-filled milkshake:

[Ballpoint Pen Drawing of a Fruit-Filled Milkshake]

![B
### Human: Thanks
### Assistant: You're welcome! Let me know if you have any more questions.
@Belluxx Belluxx changed the title After f4d277a i tried to use the config alpaca.txt but line breaks do not work Using the config alpaca.txt line breaks do not work Apr 14, 2023
@prusnak
Copy link
Collaborator

prusnak commented Apr 14, 2023

pinging @aroidzap - author of #863

@rabidcopy
Copy link
Contributor

rabidcopy commented Apr 14, 2023

Reverse prompts and interactive mode seem completely broken for me.

I'm not sure what is going on but I can't seem to use basic interactive mode with a reverse prompt anymore. It refuses to give any output and continuously hands back control to me. Disabling multiline mode doesn't seem to change this. I'm thoroughly confused. This is me repeatedly pressing enter when it keeps giving me back input and I don't know why.
./main --keep -1 -c 2048 -n -1 -m [model] -t 6 --seed 5 -b 512 -r "Human:" -i -p "a conversation between a human and an assistant."

 a conversation between a human and an assistant.\
Human: Hello?

Ass
istant
:
 Hi







!






 How




 can I assist you today?

And even without the --interactive-first flag, it doesn't even generate and immediately gives me control at the end of the starting prompt.

@prusnak
Copy link
Collaborator

prusnak commented Apr 14, 2023

Problematic PR #863 was reverted via #982

Thanks for the report!

@prusnak prusnak closed this as completed Apr 14, 2023
@brandon-lockaby
Copy link

Using bash, this works for me:

./main -m ../models/LLaMA/7B/ggml-model-f16.bin -n 4096 --threads 12 --interactive-first --color --reverse-prompt $'\n\n' -f prompts/test.txt

It's when trying --in-prefix that I find control seems to be returned to me instead of running the model

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants