Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a option to force the token end of text apears even on interative, and also shows loading porcentage #1058

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

jeffersoncgo
Copy link

No description provided.

@DannyDaemonic
Copy link
Contributor

DannyDaemonic commented Apr 20, 2023

This adds both [end of text] and [model ready] tags, so maybe another name would be better?

[end of text] will be displayed with --forceendtoken in interactive mode but [model ready] will now be displayed in interactive mode even without --forceendtoken. [model ready] is new, so it should probably only show up when using the --forceendtoken option, although, again, another name might be better.

Also, if you're looking for a "start" token, the initialization token for LLaMA models is the first space - that's my understanding of it anyway. Might want to turn that first space that's always generated into the [start of text] instead of just printing [model ready] which isn't significant.

Another thing to consider is using a switch with multiple options, like --show-signal-tokens=auto,always,never. auto could be the way it's current done, and always and never show or hide the special tokens. With proper control over them, there might even be an argument to be made that they should go to stdout instead of stderr.

I don't know how I feel about this commit as it currently is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants