Technical details about quantization

[Tunaaaaa]

Hi ggerganov,

Good morning!

I'm confused about the details of "q4_0", "q4_1", "GPTQ".
I've read the code of ./examples/quantize/quantize.cpp and found both q4_0 and q4_1 did't use the tech about "Hessian". It's more like just use min/max to do quantization.
However, at the end of ,, the youtuber video describes a method using H^-1 for optimization.
So I'm wondering what's the difference between "q4_0", "q4_1", "GPTQ" and that youtube vedio?

Thanks a lot.
Cheers,

TTTuna

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.