Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT: support openhermes 2.5 #775

Closed
Bojun-Feng opened this issue Dec 17, 2023 · 0 comments · Fixed by #776
Closed

FEAT: support openhermes 2.5 #775

Bojun-Feng opened this issue Dec 17, 2023 · 0 comments · Fixed by #776
Labels
Milestone

Comments

@Bojun-Feng
Copy link
Contributor

Is your feature request related to a problem? Please describe

For the past few months I have been hearing awesome things about the OpenHermes 2.5 model as one of the best 7B models that can outperform much larger models, especially in the r/LocalLlama community. Every week there is a new post about its capabilities.

I downloaded the Q2_K model and was very surprised (pleasantly) by its performance. Among all the 7B Q2_K models I have tried, this definitely has the best performance. It also fits inside my 8GB RAM laptop, which is another plus. I'll made a PR to integrate it into our repo.

ox7zGoygsJQFFV3rLT4v9

Some Sources include:

I find it does summaries quite well, I have asked it about a wide range of topics and it has been ~90% correct on the first response, and can kind of fall apart after going back and forth a few times, but it's only 7b.

I'll go ahead and say OpenHermes 2.5 is the best 7b model I used so far. You can say it's llama 65B and people wouldn't know.

I simply tallied the number of thumbs-ups I gave for each first (👍👍👍), second (👍👍), and third place (👍) in all three tests series - which gives us this final ranking:

1st. 👍👍👍👍👍👍👍 OpenHermes-2.5-Mistral-7B with official ChatML format
...
All three models are excellent, they're definitely the three best 7B models so far by far.

@XprobeBot XprobeBot added this to the v0.7.3 milestone Dec 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants