Replies: 1 comment 2 replies
-
|
I could be wrong but if it uses MMQ, isn't that automatically using the specialized instructions as soon as you have cuda 12.8 installed ? |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
@JohannesGaessler might be able to enlighten us on that |
Beta Was this translation helpful? Give feedback.
-
|
FP4 (hardware) is currently not used at any point in the llama.cpp/GGML code and whether or not the feature is available will make no difference. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Will llama.cpp take advantage of FP4 compute of the new chips? E.g. Blackwell GPUs.
Beta Was this translation helpful? Give feedback.
All reactions