Update custom eval loop to aid DPO debugging #770
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The documentation is not available anymore as the PR was closed or merged. |
|
Bad news @natolambert, the
|
|
That's easy to resolve though. The new approach takes ~0.005 seconds regardless of which index is used.
This also doesn't restrict us to batches anymore. We can just go for 1 sample now, for example. |
Makes it much easier to quickly read the starts of the generations
natolambert
pushed a commit
that referenced
this pull request
Sep 26, 2023
* init * run * Update custom eval loop to aid DPO debugging (#770) * sample_during_eval -> generate_during_eval * Remove unused return_tokens * Add import utils for W&B, prevent test fails * Optimize dataloader random batch selection * Separate prompt and response in logs Makes it much easier to quickly read the starts of the generations * Simplify logging * reset eval steps * manual merge fixes * revert merge * remove self.max_length * style * fix max_length --------- Co-authored-by: Tom Aarsen <37621491+tomaarsen@users.noreply.github.com>
lapp0
pushed a commit
to lapp0/trl
that referenced
this pull request
May 10, 2024
* init * run * Update custom eval loop to aid DPO debugging (huggingface#770) * sample_during_eval -> generate_during_eval * Remove unused return_tokens * Add import utils for W&B, prevent test fails * Optimize dataloader random batch selection * Separate prompt and response in logs Makes it much easier to quickly read the starts of the generations * Simplify logging * reset eval steps * manual merge fixes * revert merge * remove self.max_length * style * fix max_length --------- Co-authored-by: Tom Aarsen <37621491+tomaarsen@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello!
This is intended to be pushed directly on top of dpo_custom_eval i.e. on top of #766, but I don't have the permissions for that.
Pull Request overview
sample_during_evalis nowgenerate_during_eval- I thinksampleis a bit too vague.return_tokenswas unused, so I removed it.wandbimport without havingwandbas a mandatory dependency. I added import utils for W&B & a test.This PR is a WIP.