Skip to content
This repository has been archived by the owner on Nov 3, 2023. It is now read-only.

Add PW-Turn evaluation metrics and model comparisons to documentation #4362

Merged
merged 3 commits into from
Feb 22, 2022

Conversation

Rebecca-Qian
Copy link
Contributor

Patch description
Add documentation for using PW-Turn for different use cases and reproducing results in the human eval comparison paper:

  • How to run different model comparisons eg. BlenderBot 3B vs. Reddit 3B
  • How to change the evaluation metric, eg. selecting conversational responses that are more engaging

Testing steps
N/A

Copy link
Contributor

@EricMichaelSmith EricMichaelSmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to have! Just a bunch of nitpicky comments

Copy link
Contributor

@EricMichaelSmith EricMichaelSmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for making these revisions! Approved now. There are 2 failing CI checks, but it looks like those are both broken in main =/ No need to let them block you merging this in

@Rebecca-Qian Rebecca-Qian merged commit 369232d into main Feb 22, 2022
@Rebecca-Qian Rebecca-Qian deleted the rebeccaqian/pw-turn-documentation branch February 22, 2022 20:23
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants