Add AVX2 implementation of dequantize_row_q4_1

[slaren]

Initial tests are promising, similar gains with BLAS as with q4_0.

[slaren]

Base (no BLAS):
    60.97 seconds per pass - ETA 11.09 hours
    [1]4.4948,[2]4.9721,[3]5.8697,

BLAS:
    34.60 seconds per pass - ETA 6.29 hours
    [1]4.4305,[2]4.8844,[3]5.7737,

BLAS + AVX:
    32.06 seconds per pass - ETA 5.83 hours
    [1]4.4305,[2]4.8844,[3]5.7737,

Most of the improvement comes from BLAS, but it is still a gain.

Not sure if the lower perplexity from using BLAS is just a fluke in the first chunks, but interesting regardless.

[ggerganov]

Not sure if the lower perplexity from using BLAS is just a fluke in the first chunks, but interesting regardless.

I also observed it and it is a bit worrying because it means the non-BLAS SIMD matrix multiplication is significantly less accurate compared to BLAS. The problem is that during normal inference for text generation, after processing the prompt, we switch from BLAS to non-BLAS. Using BLAS for single token inference is terribly slow.

So I think there is a risk that we will measure too good perplexity thanks to BLAS and then in reality it will be worse due to the SIMD implementation. But I don't see a good solution less than disabling BLAS for perplexity computations.