[BUG] Issue processing NF4 double quantization

[ishan-modi]

Describe the bug

I am trying to use NF4 real quantization and came across an error because of scales not being divisible by block size. We should add padding to the scales so that we can quantize it using block quant to mitigate this issue. This can be achieved by adding reduce_block_padding function to double_quantization function.

Suggested change

scales = reduce_block_padding(
    scales.view(-1), block_sizes={-1: scale_block_size}
)

in the following line of code

Version

nvidia_modelopt == 0.27.1

[meenchen]

Thanks for the suggestion @ishan-modi ! I will make this update.

[ishan-modi]

@meenchen any updates on this ?

[ishan-modi]

@kevalmorabia97 @meenchen, when do we plan to add this ?

[kevalmorabia97]

Fix is merged. Closing this issue