🚀 A cutting-edge quantization method that outperforms current AWQ and GPTQ techniques
This repository contains several project scripts implementing the AWQ-oGPTQ large model quantization method - an advanced approach that demonstrates superior performance compared to existing AWQ and GPTQ quantization methods.
⚠️ Note: This is an ongoing project that the files are not yet fully organized.
The effectiveness of this method has been verified. However, owing to the files are not yet fully organized, you'll need to replace the corresponding scripts in the official AWQ repository and GPTQ repository in order to use AWQ-oGPTQ:
# 1. Clone or download these scripts
git clone https://github.com/BugBusterMax/AWQ-oGPTQ.git
cd AWQ-oGPTQ
# 2. Replace the corresponding scripts in the official AWQ/GPTQ installations
# 3. Compile the modified libraries