Code

🤫 TR-DPO implementation#1593

Code

Merged

kashif merged 20 commits intohuggingface:mainhuggingface/trl:mainfrom syrn1k:tr-dposyrn1k/trl:tr-dpoCopy head branch name to clipboardMay 23, 2024

+106-2

Commits

Commits on Apr 26, 2024

• 🤫 TR-DPO implementation baseline

  

  Nikita Surnachev

  committed
  6fe29e5

  Copy full SHA for 6fe29e5

• fix comments

  

  Nikita Surnachev

  committed
  a98e2ec

  Copy full SHA for a98e2ec

• docs

  

  Nikita Surnachev

  committed
  687a2e8

  Copy full SHA for 687a2e8

• fix linters

  

  Nikita Surnachev

  committed
  9f6d795

  Copy full SHA for 9f6d795

• test added

  

  Nikita Surnachev

  committed
  4e0adbc

  Copy full SHA for 4e0adbc

Commits on May 10, 2024

• Merge branch 'main' into tr-dpo

  kashif

  authored
  62de841

  Copy full SHA for 62de841

Commits on May 13, 2024

• move configs to DPOConfig

  kashif

  committed
  f918480

  Copy full SHA for f918480

• fix typo

  kashif

  committed
  f3b409e

  Copy full SHA for f3b409e

• add docs

  kashif

  committed
  d087bca

  Copy full SHA for d087bca

• fix import

  kashif

  committed
  fb01df7

  Copy full SHA for fb01df7

• use state.global_step

  kashif

  committed
  b9aabc2

  Copy full SHA for b9aabc2

• fix order of arguments

  kashif

  committed
  3b63bdd

  Copy full SHA for 3b63bdd

• make sure plugins are not none

  kashif

  committed
  247a88b

  Copy full SHA for 247a88b

Commits on May 14, 2024

• Update trl/trainer/utils.py

  Show description for 9932409
  

  kashif

  and

  BenjaminBossan

  authored
  9932409

  Copy full SHA for 9932409

• Update trl/trainer/utils.py

  Show description for e230875
  

  kashif

  and

  BenjaminBossan

  authored
  e230875

  Copy full SHA for e230875

Commits on May 15, 2024

• checking that reference model weights have changed

  kashif

  committed
  b333511

  Copy full SHA for b333511

• Merge branch 'main' into tr-dpo

  kashif

  authored
  b2b6a50

  Copy full SHA for b2b6a50

Commits on May 22, 2024

• sync_target_model as staticmethod

  kashif

  committed
  2f161b6

  Copy full SHA for 2f161b6

• set reference model

  kashif

  committed
  9f50f7d

  Copy full SHA for 9f50f7d

• Merge branch 'main' into tr-dpo

  kashif

  authored
  d16daba

  Copy full SHA for d16daba