Standardise example scripts #842
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The documentation is not available anymore as the PR was closed or merged. |
|
/benchmark-trl-experiments benchmark/benchmark_level1.sh |
|
Benchmark on Comment: succeeded ✅ |
vwxyzjn
reviewed
Oct 9, 2023
Yes, that's correct! |
vwxyzjn
approved these changes
Oct 9, 2023
|
/benchmark-trl-experiments benchmark/benchmark_level1.sh |
|
Benchmark on Comment: failed ❌ |
|
/benchmark-trl-experiments benchmark/benchmark_level1.sh |
|
Benchmark on Comment: succeeded ✅ |
|
[COSTA BENCHMARK BOT]: Here are the results |
lvwerra
approved these changes
Oct 10, 2023
Good idea! Done in a6d1d90 I'll merge if all the tests still pass |
lewtun
commented
Oct 11, 2023
|
LG! |
abhilash1910
pushed a commit
to abhilash1910/trl
that referenced
this pull request
Oct 12, 2023
* Standardise example scripts * fix plotting script * Rename run_xxx to xxx * Fix doc --------- Co-authored-by: Costa Huang <costa.huang@outlook.com>
pcuenca
pushed a commit
to huggingface/blog
that referenced
this pull request
Oct 20, 2023
lvwerra
added a commit
that referenced
this pull request
Oct 31, 2023
* enable xpu support * fix bug * review commits * fix style * add xou decorator * refactor review commit * fix test * review commit * fix test * Update benchmark.yml (#856) * Standardise example scripts (#842) * Standardise example scripts * fix plotting script * Rename run_xxx to xxx * Fix doc --------- Co-authored-by: Costa Huang <costa.huang@outlook.com> * Fix version check in import_utils.py (#853) * dont use get_peft_model if model is already peft (#857) * merge conflict * add xou decorator * resolve * resolves * upstream * refactor and precommit * fix new tests * add device mapping for xpu --------- Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com> Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> Co-authored-by: Costa Huang <costa.huang@outlook.com> Co-authored-by: Adam Pauls <adpauls@gmail.com> Co-authored-by: abhishek thakur <1183441+abhishekkrthakur@users.noreply.github.com>
lapp0
pushed a commit
to lapp0/trl
that referenced
this pull request
May 10, 2024
* Standardise example scripts * fix plotting script * Rename run_xxx to xxx * Fix doc --------- Co-authored-by: Costa Huang <costa.huang@outlook.com>
lapp0
pushed a commit
to lapp0/trl
that referenced
this pull request
May 10, 2024
* enable xpu support * fix bug * review commits * fix style * add xou decorator * refactor review commit * fix test * review commit * fix test * Update benchmark.yml (huggingface#856) * Standardise example scripts (huggingface#842) * Standardise example scripts * fix plotting script * Rename run_xxx to xxx * Fix doc --------- Co-authored-by: Costa Huang <costa.huang@outlook.com> * Fix version check in import_utils.py (huggingface#853) * dont use get_peft_model if model is already peft (huggingface#857) * merge conflict * add xou decorator * resolve * resolves * upstream * refactor and precommit * fix new tests * add device mapping for xpu --------- Co-authored-by: Leandro von Werra <lvwerra@users.noreply.github.com> Co-authored-by: lewtun <lewis.c.tunstall@gmail.com> Co-authored-by: Costa Huang <costa.huang@outlook.com> Co-authored-by: Adam Pauls <adpauls@gmail.com> Co-authored-by: abhishek thakur <1183441+abhishekkrthakur@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR standardises all the example scripts to follow the
run_xxx.pyconvention, wherexxxtypically refers to the algorithm instead of the task (i.e. have just 1 PPO example instead of calling it "sentiment tuning"). The resulting structure is as follows:IMO this makes it a bit easier for newcomers to know what each script does by filename instead of guessing whether e.g. multi adapter RL refers to PPO or something else.
I also deleted an old and duplicate multi adapter RL script
multi_adapter_rl.pywhich seems to be outdated.Eventually, we could harmonize the scripts so that the SFT and reward models produced by
run_sft.pyandrun_reward_modeling.pyare the same ones that feed intorun_ppo.pyandrun_dpo.py. This would give a true end to end pipeline that is maintained & solid for many people to work from :)