Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow usage of GPT-4 as judge model #83

Open
jaideepr97 opened this issue Jul 29, 2024 · 0 comments
Open

Allow usage of GPT-4 as judge model #83

jaideepr97 opened this issue Jul 29, 2024 · 0 comments
Assignees
@alimaredia
Labels
enhancement New feature or request mtbench Pertains to MTBench mtbenchbranch Pertains to MTBenchBranch

Comments

@jaideepr97
Copy link
Member

FastChat has native support for invoking the openAI api (when provided with a valid openAI key) to invoke GPT-4 as the judge for MT-Bench assessments. This capability should be leveraged and made available for users to access, should they wish to.
Users will need to provide their own valid openAI api key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alimaredia alimaredia

Labels
enhancement New feature or request mtbench Pertains to MTBench mtbenchbranch Pertains to MTBenchBranch
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alimaredia @jaideepr97 @nathan-weinberg