Code

IQ1_M: 1.75 bpw quantization#6302

Code

Merged

ikawrakow merged 24 commits intomasterggml-org/llama.cpp:masterfrom ik/iq1_m_newggml-org/llama.cpp:ik/iq1_m_newCopy head branch name to clipboardMar 26, 2024

Commits

Commits on Mar 25, 2024

• iq1_m: basics

  Kawrakow

  committed
  2a2d66d

  Copy full SHA for 2a2d66d

• iq1_m: basics-2

  Kawrakow

  committed
  ac8b3dd

  Copy full SHA for ac8b3dd

• iq1_m: CUDA dequantize works

  Show description for 1df37b6
  Kawrakow

  committed
  1df37b6

  Copy full SHA for 1df37b6

• iq1_m: separate shifts for each group of 8 in a block

  Show description for 282f278
  Kawrakow

  committed
  282f278

  Copy full SHA for 282f278

• iq1_m: go to 3-bit scales

  Show description for 308c50d
  Kawrakow

  committed
  308c50d

  Copy full SHA for 308c50d

• iq1_m: scalar dot product

  Kawrakow

  committed
  64b9dfd

  Copy full SHA for 64b9dfd

• iq1_m: AVX2 dot product

  Kawrakow

  committed
  a139de5

  Copy full SHA for a139de5

• iq1_m: very slightly faster AVX2 dot product

  Kawrakow

  committed
  379fdb6

  Copy full SHA for 379fdb6

• iq1_m: ARM_NEON dot product

  Show description for 8009b6d
  Kawrakow

  committed
  8009b6d

  Copy full SHA for 8009b6d

• iq1_m: Metal - dequantize works, dot product does not

  Kawrakow

  committed
  0e36afa

  Copy full SHA for 0e36afa

• iq1_m: Metal now works

  Show description for 19fb974
  Kawrakow

  committed
  19fb974

  Copy full SHA for 19fb974

• iq1_m: minor

  Kawrakow

  committed
  abc1d4f

  Copy full SHA for abc1d4f

• iq1_m: checking pure iq1_m quantization

  Show description for dff85a8
  Kawrakow

  committed
  dff85a8

  Copy full SHA for dff85a8

• iiq1_m: slightly faster ARM_NEON dot product

  Show description for f664692
  Kawrakow

  committed
  f664692

  Copy full SHA for f664692

• iq1_m: faster ARM_NEON dot product

  Show description for b1d1c26
  Kawrakow

  committed
  b1d1c26

  Copy full SHA for b1d1c26

• iq1_m: another minor ARM_NEON dot product improvement

  Show description for 78ce561
  Kawrakow

  committed
  78ce561

  Copy full SHA for 78ce561

• iq1_m: small PPL improvement via super-block scale adjustment

  Show description for 3d9c21f
  Kawrakow

  committed
  3d9c21f

  Copy full SHA for 3d9c21f

• iq1_m: adapt to CUDA refactoring

  Kawrakow

  committed
  480d6d6

  Copy full SHA for 480d6d6

• iq1_m: remove unused variable

  Show description for 62dd11f
  Kawrakow

  committed
  62dd11f

  Copy full SHA for 62dd11f

• iq1_m: add to backend-ops tests

  Kawrakow

  committed
  22fa121

  Copy full SHA for 22fa121

Commits on Mar 26, 2024

• iq1_m: fix Windows ARM

  Kawrakow

  committed
  b68f32b

  Copy full SHA for b68f32b

• iq1_m: use common definition of iq1m_scale_t

  Kawrakow

  committed
  9a5786e

  Copy full SHA for 9a5786e

• cuda: assert -> NO_DEVICE_CODE

  Kawrakow

  committed
  cdb2d65

  Copy full SHA for cdb2d65

• iq1_M: PR comments

  Kawrakow

  committed
  6e4cef5

  Copy full SHA for 6e4cef5