Code

Add JSD Loss for Distillation#425

Code

Draft

austin362667 wants to merge 4 commits intomainlinkedin/Liger-Kernel:mainfrom austin362667/chunked_compiled_jsd_losslinkedin/Liger-Kernel:austin362667/chunked_compiled_jsd_lossCopy head branch name to clipboard

+787

Commits

Commits on Dec 17, 2024

• Add chunked and compiled JSD loss

  Show description for ffb416b
  

  austin362667

  authored and

  shivam15s

  committed
  ffb416b

• Add tests for JSD loss

  Show description for 82c80fe
  

  austin362667

  authored and

  shivam15s

  committed
  82c80fe

• Add benchmarks for JSD loss

  Show description for 7b22ac7
  

  austin362667

  authored and

  shivam15s

  committed
  7b22ac7

• Set JSD loss default chunk_size to 2

  Show description for 1d3b064
  

  austin362667

  authored and

  shivam15s

  committed
  1d3b064