[ENH] add internlm2-chat-20b-ppo #207
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
YannDubs
reviewed
Jan 16, 2024
YannDubs
reviewed
Jan 16, 2024
| torch_dtype: "bfloat16" | ||
| trust_remote_code: True | ||
| is_fast_tokenizer: False | ||
| max_new_tokens: 2048 |
There was a problem hiding this comment.
note that we typically use 4096 for newer models. 2048 is perfectly fine just saying in case some outputs are truncated
There was a problem hiding this comment.
Thanks for your advice!
Currently we haven't observed too many truncated completions. We suppose 2048 tokens is enough for our model. Responses above 2048 tokens are generally nonsense repetition, which might waste GPU time & GPT tokens
|
Very impressive results @C1rN09, I'll merge once you remove the reference_outputs! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi, we are releasing internlm2 this week and here is our result on AlpacaEval.
The huggingface link is private now and will be public after its release. Hope to merge ASAP :)
Extra Modifications
The
eostoken,[UNUSED_TOKEN_145]for our model, seems to appear in the completion results. It might spoil the final score. So I added aremove_endingargument tohuggingface_local_completionsto strip this annoying word.If this is not preferred, please let me know and we can work out another solution!