Don't force immediate interactive without -i
#354
Conversation
|
I think it's good to not force interactive mode immediately (in fact that was how it worked when I first made the patch, but the logic seems to have changed at some point), but in this combination the flags seem to be rendered a bit misleading. What I conceived of in the beginning:
The scenario you are proposing:
I think (2) is a valid use case, and moreover it is confusing that the "interactive mode" flag Rather than loading |
|
@blackhole89 I agree 100% with you that the first scenario is the most intuitive and useful (I'll do the changes). I did not remove --interactive-first (but I do remember seeing it in a previous build; no idea what happened to it). |
|
@tjohnman Thanks! Wasn't meaning to imply you had anything to do with the removal - development has been moving quickly and chaotically, it probably just fell on the wayside in some refactoring along the way. |
Sometimes we might want to use a reverse prompt but we want to let the model generate tokens right after the initial prompt. So we don't force user input mode if the -i flag wasn't specified and instead let it run until we encounter the reverse prompt. This gives use some more flexibility, since it doesn't force the user to enter a newline if they want to let the model generate text right after the initial prompt and only be asked for input if the reverse prompt is encountered. The `--interactive-first` flag is reintroduced to force the old behavior. `-r` behaves like `-i` plus introduces a reverse prompt (it can be specified more than once).
tjohnman
force-pushed
the
no-forced-interactive-start
branch
from
77d9a8a to
98570dd
Compare
|
this kind of broke instruction mode. this change needs to be only for |
| @@ -1032,7 +1036,7 @@ int main(int argc, char ** argv) { | |||
| #endif | |||
| " - Press Return to return control to LLaMa.\n" | |||
| " - If you want to submit another line, end your input in '\\'.\n\n"); | |||
| is_interacting = true; | |||
| is_interacting = params.interactive_start; | |||
There was a problem hiding this comment.
params.interactive_start || params.instruct
Sometimes we might want to use a reverse prompt but we want to let the model generate tokens right after the initial prompt. So we don't force user input mode if the
-iflag wasn't specified and instead let it run until we encounter the reverse prompt.This gives use some more flexibility, since it doesn't force the user to enter a newline if they want to let the model generate text right after the initial prompt and only be asked for input if the reverse prompt is encountered.