Code

Support calibrating kv cache scales#17

Code

Merged

mgoin merged 10 commits intomainneuralmagic/AutoFP8:mainfrom support-kv-cache-scalesneuralmagic/AutoFP8:support-kv-cache-scalesCopy head branch name to clipboardJun 18, 2024

+249-69

Commits

Commits on Jun 14, 2024

• Support calibrating kv cache scales

  mgoin

  committed
  e325874

• Add comment

  mgoin

  committed
  4eae078

• Fix weight name

  mgoin

  committed
  084feb8

Commits on Jun 15, 2024

• Add Qwen test

  mgoin

  committed
  f3a29ae

Commits on Jun 16, 2024

• Fix kv cache test count

  mgoin

  committed
  e620300

• Add fixed target sizes

  mgoin

  committed
  817cf9c

• Fix proj linear count

  mgoin

  committed
  142eac8

Commits on Jun 17, 2024

• Merge branch 'main' into support-kv-cache-scales

  mgoin

  authored
  bb15a62

Commits on Jun 18, 2024

• Switch from output_scale to kv_scale

  mgoin

  committed
  8556c86

• Add example

  mgoin

  committed
  65c4e83