[BUG] 3bit quantization not working

[sidhantls]

Describe the bug
I'm trying to quantize LLM to 3 bits. However, the quantization code runs with an error at the end. Yet, when I set bits=4 for the same code, it works.

Software Info
Windows 10, Python 3.10

Torch: Version: 2.6.0+cu124, transformers=4.48.2, accelerate=1.3.0

To Reproduce

from datasets import load_dataset
from gptqmodel import GPTQModel, QuantizeConfig

model_id = "EleutherAI/pythia-160m"

calibration_dataset = load_dataset(
    "allenai/c4",
    data_files="en/c4-train.00001-of-01024.json.gz",
    split="train"
  ).select(range(1024))["text"]

calibration_dataset = [" ".join(item.split()[:30]) for item in calibration_dataset] # speedup

quantize_config = QuantizeConfig(
    bits=3, # works with bit=4
    group_size=128,
)
model = GPTQModel.load(model_id, quantize_config)

# increase `batch_size` to match gpu/vram specs to speed up quantization
model.quantize(calibration_dataset, batch_size=1)
model.save("saved_model")

Error:

File ~\Desktop\GPTQModel\gptqmodel\utils\model.py:440, in pack_module(name, qModules, quantizers, layers, pbar)
    433 qModules[name].to(CPU)
    434 layers[name], scale, zero, g_idx = (
    435     layers[name].to(CPU),
    436     scale.to(CPU),
    437     zero.to(CPU),
    438     g_idx.to(CPU) if g_idx is not None else None,
    439 )
--> 440 qModules[name].pack(layers[name], scale, zero, g_idx)
    441 qModules[name].to(layer_device)
    442 if pbar:

File ~\Desktop\GPTQModel\gptqmodel\nn_modules\qlinear\__init__.py:320, in PackableQuantLinear.pack(self, linear, scales, zeros, g_idx)
    318 qweight[row] |= intweight[row_offset_plus_10] << 31
    319 row += 1
--> 320 qweight[row] |= (intweight[row_offset_plus_10] >> 1) & 0x3
    321 for j in range(10):
    322     qweight[row] |= intweight[row_offset + j] << (3 * j + 2)

IndexError: index 72 is out of bounds for axis 0 with size 72