What is better 13B q4_0 or 7B q8_0 ??? #1468
skidd-level-100
started this conversation in
General
Replies: 1 comment
-
|
In terms of perplexity q4_0 is always better than the 16bit version of the lower model. There are some examples in the readme and there's a discussion with comparisons for each model. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I would like to know what kind of model is best for its parameter size,
based on file size I would guess that the 13B one is better.
please let me know
Beta Was this translation helpful? Give feedback.
All reactions