index unpack problem

[laomao0]

Hi, there is a small issue in https://github.com/microsoft/VPTQ/blob/main/vptq/layers/vqlinear.py#L519 function,

Line531-539

            # print(f'self.indices shape: {self.indices.shape}')
            indices, res_indices = self.unpack_index_tensor(
                pack_tensor=self.indices,
                index_bits=index_bits,
                num_elements=self.in_features,
                res_bits=index_res_bits,
                num_res_elements=self.in_features,
                index_dtype=torch.uint16,
            )

should be

            # print(f'self.indices shape: {self.indices.shape}')
            indices, res_indices = self.unpack_index_tensor(
                pack_tensor=self.indices,
                index_bits=index_bits,
                num_elements=self.group_size, #here changed
                res_bits=index_res_bits,
                num_res_elements=self.group_size, #here changed
                index_dtype=torch.uint16,
            )

since once the weight is grouped, the packed size from inchannel need also be divided.

[YangWang92]

cool, you can raise an issue as a contributor of the project~

[YangWang92]

fixed in #103

Thanks!