Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] add internlm2-chat-20b-ppo #207

Merged
merged 6 commits into from
Jan 16, 2024
Merged

[ENH] add internlm2-chat-20b-ppo #207

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d7b97d0
add model config
C1rN09 Jan 16, 2024
2153360
modify `huggingface_local_completion` to remove EOS
C1rN09 Jan 16, 2024
7081250
add results & update leaderboard
C1rN09 Jan 16, 2024
e8b579a
delete extra leaderboard csv
C1rN09 Jan 16, 2024
0efc893
add docstring of `remove_ending`
C1rN09 Jan 16, 2024
4a816e3
remove reference_outputs.json
C1rN09 Jan 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4,832 changes: 4,832 additions & 0 deletions results/internlm2-chat-20b-ppo/model_outputs.json
View file
Open in desktop

Large diffs are not rendered by default.

64,151 changes: 64,151 additions & 0 deletions results/internlm2-chat-20b-ppo/weighted_alpaca_eval_gpt4_turbo/annotations.json
View file
Open in desktop

Large diffs are not rendered by default.

9 changes: 8 additions & 1 deletion src/alpaca_eval/decoders/huggingface_local.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ def huggingface_local_completions(
batch_size: int = 1,
model_kwargs=None,
cache_dir: Optional[str] = constants.DEFAULT_CACHE_DIR,
remove_ending: Optional[str] = None,
is_fast_tokenizer: bool = True,
adapters_name: Optional[str] = None,
**kwargs,
Expand All @@ -58,6 +59,9 @@ def huggingface_local_completions(
cache_dir : str, optional
Directory to use for caching the model.

remove_ending : str, optional
The ending string to be removed from completions. Typically eos_token.

kwargs :
Additional kwargs to pass to `InferenceApi.__call__`.
"""
Expand Down Expand Up @@ -143,7 +147,10 @@ def huggingface_local_completions(
pad_token_id=tokenizer.pad_token_id,
)
):
completions.append(out[0]["generated_text"])
generated_text = out[0]["generated_text"]
if remove_ending is not None and generated_text.endswith(remove_ending):
generated_text = generated_text[:-len(remove_ending)]
completions.append(generated_text)

logging.info(f"Time for {n_examples} completions: {t}")

Expand Down
1 change: 1 addition & 0 deletions ...lpaca_eval/leaderboards/data_AlpacaEval_2/weighted_alpaca_eval_gpt4_turbo_leaderboard.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ gpt4,23.576789314782605,1.275704201206918,179,618,8,805,22.732919254658384,minim
gpt4_0314,22.07325892871952,1.246672549460114,172,627,6,805,21.73913043478261,verified,1371
mistral-medium,21.855772543461345,1.2682402187234831,164,639,2,805,20.496894409937887,minimal,1500
xwinlm-70b-v0.1,21.812957073994184,1.2303274476035724,166,635,4,805,20.869565217391305,community,1775
internlm2-chat-20b-ppo,21.749154500562636,1.2443662409556862,170,632,3,805,21.304347826086957,community,2373
evo-v2-7b,20.83411302254932,1.2159901798147486,158,644,3,805,19.81366459627329,community,1754
pairrm-tulu-2-70b,18.63896296749771,1.1924966699965729,140,665,0,805,17.391304347826086,community,1607
Mixtral-8x7B-Instruct-v0.1,18.25531762637268,1.1885585968848205,135,668,2,805,16.8944099378882,minimal,1465
Expand Down
15 changes: 15 additions & 0 deletions src/alpaca_eval/models_configs/internlm2-chat-20b-ppo/configs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
internlm2-chat-20b-ppo:
prompt_template: "internlm2-chat-20b-ppo/prompt.txt"
fn_completions: "huggingface_local_completions"
completions_kwargs:
model_name: "internlm/internlm2-chat-20b"
model_kwargs:
torch_dtype: "bfloat16"
trust_remote_code: True
is_fast_tokenizer: False
max_new_tokens: 2048
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that we typically use 4096 for newer models. 2048 is perfectly fine just saying in case some outputs are truncated

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your advice!

Currently we haven't observed too many truncated completions. We suppose 2048 tokens is enough for our model. Responses above 2048 tokens are generally nonsense repetition, which might waste GPU time & GPT tokens

batch_size: 32
eos_token_id: 92542
remove_ending: "[UNUSED_TOKEN_145]"
pretty_name: "InternLM2 Chat 20B ppo"
link: "https://huggingface.co/internlm/internlm2-chat-20b"
3 changes: 3 additions & 0 deletions src/alpaca_eval/models_configs/internlm2-chat-20b-ppo/prompt.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[UNUSED_TOKEN_146]user
{instruction}[UNUSED_TOKEN_145]
[UNUSED_TOKEN_146]assistant