Code

Implement BoN for training and eval#528

Code

Open

Dahoas wants to merge 40 commits intomainCarperAI/trlx:mainfrom BoNCarperAI/trlx:BoNCopy head branch name to clipboard

+185-58

Commits

Commits on Jul 18, 2023

• Implementing support for dense rewards

  Dahoas

  committed
  f04446d

  Copy full SHA for f04446d

• added "num_return_sequences" param which corresponds to n in Best-of-N sampling

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  13a01fc

  Copy full SHA for 13a01fc

• updates to "num_return_sequences" param

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  5421a73

  Copy full SHA for 5421a73

• BoN implementation

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  2f3ac28

  Copy full SHA for 2f3ac28

• Changed back to default.

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  2f1dace

  Copy full SHA for 2f1dace

• TopK sampling instead of Top1

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  f58170d

  Copy full SHA for f58170d

• summed along dim=1

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  be8bc1a

  Copy full SHA for be8bc1a

• Generating samples in chunks

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  608d812

  Copy full SHA for 608d812

• added gen_chunk_size parameter

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  d8557e7

  Copy full SHA for d8557e7

• chunking in forward prop

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  8ef9c36

  Copy full SHA for 8ef9c36

• chunking generations in train and eval

  

  SharathRaparthy

  authored and

  Dahoas

  committed
  4c1d82d

  Copy full SHA for 4c1d82d

• Implementing support for dense rewards

  Dahoas

  committed
  ecd5107

  Copy full SHA for ecd5107

• Fix distributed ref_mean, ref_var bug for dense rewards

  Dahoas

  committed
  4071604

  Copy full SHA for 4071604

• Make generation respect max seq length

  Dahoas

  committed
  5f41413

  Copy full SHA for 5f41413

• Make experience before first round of training

  Dahoas

  committed
  22ae83f

  Copy full SHA for 22ae83f

• Refactoring .generate/.generate_eval

  Dahoas

  committed
  7d0a4be

  Copy full SHA for 7d0a4be

• Fix BoN metric support

  Dahoas

  committed
  b79dd19

  Copy full SHA for b79dd19

• Enforce chunk_size param for eval generation when present

  Dahoas

  committed
  cb49dc5

  Copy full SHA for cb49dc5

• Fix: Don't shuffle prompt dataset

  Dahoas

  committed
  e290412

  Copy full SHA for e290412

• Move inputs to device

  Dahoas

  committed
  391d04c

  Copy full SHA for 391d04c

• Fix style

  Dahoas

  committed
  8de84e4

  Copy full SHA for 8de84e4

Commits on Jul 21, 2023

• Fix chunked generation

  Dahoas

  committed
  3d7e0d5

  Copy full SHA for 3d7e0d5

Commits on Jul 22, 2023

• fix(accelerate_base_trainer): order of keyword arguments

  maxreciprocate

  authored
  1fda0ce

  Copy full SHA for 1fda0ce

Commits on Aug 7, 2023

• Merging main

  Dahoas

  committed
  4ac1707

  Copy full SHA for 4ac1707

• Merge branch 'BoN' of https://github.com/CarperAI/trlx into BoN

  Dahoas

  committed
  de3d854

  Copy full SHA for de3d854

• Removing old example

  Dahoas

  committed
  3ce3c2b

  Copy full SHA for 3ce3c2b

• Fix: remove extraneous method args

  Dahoas

  committed
  2635de5

  Copy full SHA for 2635de5

• Fix: Always set generate_experience_kwargs

  Dahoas

  committed
  1be2c3c

  Copy full SHA for 1be2c3c

• Fix: Remove mask from RunningMoments update call

  Dahoas

  committed
  3cba0db

  Copy full SHA for 3cba0db

• Fix: style

  Dahoas

  committed
  0cb91c4

  Copy full SHA for 0cb91c4

• Fix: rename 'gen_chunk_size' to 'chunk_size'

  Dahoas

  committed
  cc92911

  Copy full SHA for cc92911

• Fix: generated samples padding

  Dahoas

  committed
  4297f98

  Copy full SHA for 4297f98

• Remove prints

  Dahoas

  committed
  36f06af

  Copy full SHA for 36f06af

Commits on Aug 21, 2023

• Rename 'num_train_sequences' to 'num_topk_samples'

  Dahoas

  committed
  a2980dd

  Copy full SHA for a2980dd

• Address nits

  Dahoas

  committed
  3d5a639

  Copy full SHA for 3d5a639

• Fix: style

  Dahoas

  committed
  87837b6

  Copy full SHA for 87837b6

• Set 'num_return_sequences' to 1 by default

  Dahoas

  committed
  ed93be8

  Copy full SHA for ed93be8

• Fix: typo

  Dahoas

  committed
  24925c8

  Copy full SHA for 24925c8

Commits on Sep 1, 2023

• Merge branch 'main' into BoN

  maxreciprocate

  committed
  a022d3f

  Copy full SHA for a022d3f

• Merge branch 'main' into bon-x

  maxreciprocate

  committed
  9680c9f

  Copy full SHA for 9680c9f