Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Higgs Llama3-70B V2 Results #367

Merged
merged 2 commits into from
Jul 17, 2024

Conversation

sxjscience
Copy link
Contributor

Higgs-Llama-3-70B-V2 is an improved version over bosonai/Higgs-Llama-3-70B. We have improved its reasoning and roleplaying capability.

Leaderboard:

                            length_controlled_winrate  win_rate  standard_error  n_total  avg_length
gpt-4o-2024-05-13                               57.46     51.33            1.47      805        1873
higgs-llama-3-70b-v2                            56.76     68.64            1.32      805        2657
gpt-4-turbo-2024-04-09                          55.02     46.12            1.47      805        1802
gpt4_1106_preview                               50.00     50.00            0.00      805        2049
claude-3-opus-20240229                          40.51     29.11            1.39      805        1388
claude-3-sonnet-20240229                        34.87     25.56            1.34      805        1420
Meta-Llama-3-70B-Instruct                       34.42     33.18            1.39      805        1919
gemini-pro                                      24.38     18.18            1.16      805        1456
Mixtral-8x7B-Instruct-v0.1                      23.69     18.26            1.19      805        1465
Meta-Llama-3-8B-Instruct                        22.92     22.57            1.26      805        1899
Mistral-7B-Instruct-v0.2                        17.11     14.72            1.08      805        1676
alpaca-7b                                        5.88      2.59            0.49      805         396

@YannDubs
Copy link
Collaborator

very impressive results @sxjscience ! outputs seem pretty long but it seems to still be liked by AlpacaEval LC

@YannDubs YannDubs merged commit e3b5243 into tatsu-lab:main Jul 17, 2024
2 checks passed
@sxjscience
Copy link
Contributor Author

Thanks @YannDubs . I also pinged you on Discord to see if you may be able to verify the results. (Sorry for pinging you here also.).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants