Code

Gkd trainer#1814

Code

Merged

kashif merged 176 commits intohuggingface:mainhuggingface/trl:mainfrom kashif:gkd-trainerkashif/trl:gkd-trainerCopy head branch name to clipboardSep 11, 2024

+940

Commits

Commits on Jul 5, 2024

• initial

  kashif

  committed
  4124e0b

  Copy full SHA for 4124e0b

• Merge remote-tracking branch 'upstream/main' into gkd-trainer

  kashif

  committed
  afc9a89

  Copy full SHA for afc9a89

Commits on Jul 7, 2024

• initial gkd script

  kashif

  committed
  fc62694

  Copy full SHA for fc62694

Commits on Jul 8, 2024

• fix output dir name

  kashif

  committed
  80981df

  Copy full SHA for 80981df

• smaller max_new_tokens_response size

  kashif

  committed
  ea37359

  Copy full SHA for ea37359

Commits on Jul 9, 2024

• fix tab

  kashif

  committed
  141868d

  Copy full SHA for 141868d

Commits on Jul 10, 2024

• Merge remote-tracking branch 'upstream/main' into gkd-trainer

  kashif

  committed
  136c77b

  Copy full SHA for 136c77b

• use temperature from config

  kashif

  committed
  fd6f35d

  Copy full SHA for fd6f35d

Commits on Jul 11, 2024

• initial docs

  kashif

  committed
  4a5c55b

  Copy full SHA for 4a5c55b

• initial test

  kashif

  committed
  7b30d36

  Copy full SHA for 7b30d36

• add generalized_jsd_loss

  kashif

  committed
  3d3636e

  Copy full SHA for 3d3636e

• some docs

  kashif

  committed
  4a6e9e0

  Copy full SHA for 4a6e9e0

• fix order of interpolation

  kashif

  committed
  1594d95

  Copy full SHA for 1594d95

• use log_target=True

  kashif

  committed
  061e4e3

  Copy full SHA for 061e4e3

Commits on Jul 15, 2024

• Merge remote-tracking branch 'upstream/main' into gkd-trainer

  kashif

  committed
  f2e7ef7

  Copy full SHA for f2e7ef7

• fix formatting

  kashif

  committed
  6e22034

  Copy full SHA for 6e22034

• docstrings

  kashif

  committed
  119ad0d

  Copy full SHA for 119ad0d

• add peft example

  kashif

  committed
  371ac65

  Copy full SHA for 371ac65

• more docs

  kashif

  committed
  2ecd2fa

  Copy full SHA for 2ecd2fa

• formatting

  kashif

  committed
  9d62303

  Copy full SHA for 9d62303

• fix ordering

  kashif

  committed
  07250ac

  Copy full SHA for 07250ac

• use unwrap_model_for_generation

  kashif

  committed
  87b0e20

  Copy full SHA for 87b0e20

Commits on Jul 17, 2024

• initial DataCollatorForLastCompletionLM

  kashif

  committed
  b975de1

  Copy full SHA for b975de1

Commits on Jul 21, 2024

• Merge remote-tracking branch 'upstream/main' into gkd-trainer

  kashif

  committed
  a950fc8

  Copy full SHA for a950fc8

• add generation inputs

  kashif

  committed
  10889be

  Copy full SHA for 10889be

• logits from the completions

  kashif

  committed
  caceb8e

  Copy full SHA for caceb8e

Commits on Jul 23, 2024

• add eps to probs

  kashif

  committed
  afb00d5

  Copy full SHA for afb00d5

Commits on Aug 6, 2024

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  797df92

  Copy full SHA for 797df92

• select the logits after removing the padding

  kashif

  committed
  d959542

  Copy full SHA for d959542

• formatting

  kashif

  committed
  f37a6e5

  Copy full SHA for f37a6e5

• interpolate log_probs

  kashif

  committed
  8121d83

  Copy full SHA for 8121d83

• add back online sampling

  kashif

  committed
  e792d39

  Copy full SHA for e792d39

Commits on Aug 9, 2024

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  194ac95

  Copy full SHA for 194ac95

• update tests

  kashif

  committed
  75b0f35

  Copy full SHA for 75b0f35

• fix typos

  kashif

  committed
  1f6531c

  Copy full SHA for 1f6531c

Commits on Aug 13, 2024

• Merge branch 'main' into gkd-trainer

  lewtun

  authored
  549cada

  Copy full SHA for 549cada

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  1ec5bb7

  Copy full SHA for 1ec5bb7

• Update docs/source/gkd_trainer.md

  Show description for e60c1d4
  

  kashif

  and

  lewtun

  authored
  e60c1d4

  Copy full SHA for e60c1d4

• Update docs/source/gkd_trainer.md

  Show description for e9e3674
  

  kashif

  and

  lewtun

  authored
  e9e3674

  Copy full SHA for e9e3674

• Update docs/source/gkd_trainer.md

  Show description for 48a081c
  

  kashif

  and

  lewtun

  authored
  48a081c

  Copy full SHA for 48a081c

• Update docs/source/gkd_trainer.md

  Show description for 8f98bc2
  

  kashif

  and

  lewtun

  authored
  8f98bc2

  Copy full SHA for 8f98bc2

• Update docs/source/gkd_trainer.md

  Show description for d64b4cb
  

  kashif

  and

  lewtun

  authored
  d64b4cb

  Copy full SHA for d64b4cb

• Update docs/source/gkd_trainer.md

  Show description for f89118d
  

  kashif

  and

  lewtun

  authored
  f89118d

  Copy full SHA for f89118d

• Update docs/source/_toctree.yml

  Show description for 599e069
  

  kashif

  and

  lewtun

  authored
  599e069

  Copy full SHA for 599e069

• Update examples/scripts/gkd.py

  Show description for bd395fc
  

  kashif

  and

  lewtun

  authored
  bd395fc

  Copy full SHA for bd395fc

• Update examples/scripts/gkd.py

  Show description for c2fc046
  

  kashif

  and

  lewtun

  authored
  c2fc046

  Copy full SHA for c2fc046

• Update examples/scripts/gkd.py

  Show description for 6f2e899
  

  kashif

  and

  lewtun

  authored
  6f2e899

  Copy full SHA for 6f2e899

• Update examples/scripts/gkd.py

  Show description for b5f74a1
  

  kashif

  and

  lewtun

  authored
  b5f74a1

  Copy full SHA for b5f74a1

• Update examples/scripts/gkd.py

  Show description for cc0f187
  

  kashif

  and

  lewtun

  authored
  cc0f187

  Copy full SHA for cc0f187

• Update examples/scripts/gkd.py

  Show description for 967f7e4
  

  kashif

  and

  lewtun

  authored
  967f7e4

  Copy full SHA for 967f7e4

• Update examples/scripts/gkd.py

  Show description for 3a5c348
  

  kashif

  and

  lewtun

  authored
  3a5c348

  Copy full SHA for 3a5c348

• Update examples/scripts/gkd.py

  Show description for 1f6024a
  

  kashif

  and

  lewtun

  authored
  1f6024a

  Copy full SHA for 1f6024a

• use Qwen2

  kashif

  committed
  53185ee

  Copy full SHA for 53185ee

• Update trl/trainer/gkd_config.py

  Show description for a557fc7
  

  kashif

  and

  lewtun

  authored
  a557fc7

  Copy full SHA for a557fc7

• Update trl/trainer/gkd_config.py

  Show description for 740d4af
  

  kashif

  and

  lewtun

  authored
  740d4af

  Copy full SHA for 740d4af

• Update trl/trainer/gkd_trainer.py

  Show description for e53a590
  

  kashif

  and

  lewtun

  authored
  e53a590

  Copy full SHA for e53a590

• Update trl/trainer/gkd_trainer.py

  Show description for cd161ac
  

  kashif

  and

  lewtun

  authored
  cd161ac

  Copy full SHA for cd161ac

• Update trl/trainer/gkd_trainer.py

  Show description for 03a860c
  

  kashif

  and

  lewtun

  authored
  03a860c

  Copy full SHA for 03a860c

• Update trl/trainer/gkd_trainer.py

  Show description for dd157ea
  

  kashif

  and

  lewtun

  authored
  dd157ea

  Copy full SHA for dd157ea

• Update trl/trainer/gkd_trainer.py

  Show description for 9f84f69
  

  kashif

  and

  lewtun

  authored
  9f84f69

  Copy full SHA for 9f84f69

• Update trl/trainer/gkd_trainer.py

  Show description for f728f80
  

  kashif

  and

  lewtun

  authored
  f728f80

  Copy full SHA for f728f80

• Update tests/test_gkd_trainer.py

  Show description for ce900f4
  

  kashif

  and

  lewtun

  authored
  ce900f4

  Copy full SHA for ce900f4

• Update trl/trainer/gkd_trainer.py

  Show description for e08f83a
  

  kashif

  and

  lewtun

  authored
  e08f83a

  Copy full SHA for e08f83a

• fixes

  kashif

  committed
  8db3133

  Copy full SHA for 8db3133

• renamed lamda to lmbda due to keyword

  kashif

  committed
  32bfcf8

  Copy full SHA for 32bfcf8

Commits on Aug 14, 2024

• fix config name

  kashif

  committed
  8b6e3f7

  Copy full SHA for 8b6e3f7

• move collator to utils

  kashif

  committed
  7332a42

  Copy full SHA for 7332a42

• fix formatting

  kashif

  committed
  536d00e

  Copy full SHA for 536d00e

• Update trl/trainer/gkd_trainer.py

  Show description for a96b072
  

  kashif

  and

  lewtun

  authored
  a96b072

  Copy full SHA for a96b072

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  e60b625

  Copy full SHA for e60b625

• Update trl/trainer/gkd_trainer.py

  Show description for cc05927
  

  kashif

  and

  lewtun

  authored
  cc05927

  Copy full SHA for cc05927

Commits on Aug 15, 2024

• the larger the lmbda the more on policy it should be

  kashif

  committed
  89b9836

  Copy full SHA for 89b9836

• Use JSD instead of KL

  kashif

  committed
  b7f775c

  Copy full SHA for b7f775c

Commits on Aug 16, 2024

• use DataCollatorForChatML

  kashif

  committed
  3a403d2

  Copy full SHA for 3a403d2

• fix labels

  kashif

  committed
  e69f7f3

  Copy full SHA for e69f7f3

• use torch_call

  kashif

  committed
  bdac38e

  Copy full SHA for bdac38e

Commits on Aug 19, 2024

• Update examples/scripts/gkd.py

  Show description for 2623af1
  

  kashif

  and

  lewtun

  authored
  2623af1

  Copy full SHA for 2623af1

• Update examples/scripts/gkd.py

  Show description for 2bfecbb
  

  kashif

  and

  lewtun

  authored
  2bfecbb

  Copy full SHA for 2bfecbb

• Update examples/scripts/gkd.py

  Show description for 4694d5e
  

  kashif

  and

  lewtun

  authored
  4694d5e

  Copy full SHA for 4694d5e

• Update examples/scripts/gkd.py

  Show description for 8d4ca0c
  

  kashif

  and

  lewtun

  authored
  8d4ca0c

  Copy full SHA for 8d4ca0c

• Update examples/scripts/gkd.py

  Show description for afee23a
  

  kashif

  and

  lewtun

  authored
  afee23a

  Copy full SHA for afee23a

• Update examples/scripts/gkd.py

  Show description for eab9fb8
  

  kashif

  and

  lewtun

  authored
  eab9fb8

  Copy full SHA for eab9fb8

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  ffe018b

  Copy full SHA for ffe018b

Commits on Aug 20, 2024

• set default collator to DataCollatorForChatML

  kashif

  committed
  63016bd

  Copy full SHA for 63016bd

• return only the prompts

  kashif

  committed
  fcb7640

  Copy full SHA for fcb7640

• fix labels of generated outputs

  kashif

  committed
  ca6f75e

  Copy full SHA for ca6f75e

• formatting

  kashif

  committed
  a1dd6ea

  Copy full SHA for a1dd6ea

• fix comment

  kashif

  committed
  1606ca7

  Copy full SHA for 1606ca7

• add missing _prepare_deepspeed

  kashif

  committed
  b77dc7e

  Copy full SHA for b77dc7e

• no attention mask when generating

  kashif

  committed
  9e90db1

  Copy full SHA for 9e90db1

• update test

  kashif

  committed
  90f87b8

  Copy full SHA for 90f87b8

• set a sensible max_seq_length

  kashif

  committed
  3432d57

  Copy full SHA for 3432d57

• set default in the collator

  kashif

  committed
  8749ee5

  Copy full SHA for 8749ee5

Commits on Aug 27, 2024

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  6a6c30e

  Copy full SHA for 6a6c30e

• Update tests/test_gkd_trainer.py

  Show description for 22cca65
  

  kashif

  and

  qgallouedec

  authored
  22cca65

  Copy full SHA for 22cca65

• Update tests/test_gkd_trainer.py

  Show description for f9111e7
  

  kashif

  and

  qgallouedec

  authored
  f9111e7

  Copy full SHA for f9111e7

• fix padding

  kashif

  committed
  5ebe2b5

  Copy full SHA for 5ebe2b5

• formatting

  kashif

  committed
  a2fa597

  Copy full SHA for a2fa597

• Update tests/test_gkd_trainer.py

  Show description for 1925ea4
  

  kashif

  and

  qgallouedec

  authored
  1925ea4

  Copy full SHA for 1925ea4

• fix tests

  kashif

  committed
  b5232b6

  Copy full SHA for b5232b6

• TestGeneralizedJSDLoss

  kashif

  committed
  560940d

  Copy full SHA for 560940d

• fix typos

  kashif

  committed
  4b5eedc

  Copy full SHA for 4b5eedc

• use a mask to calculate jsd loss

  kashif

  committed
  4fe09cf

  Copy full SHA for 4fe09cf

• use the super() training_step after the inputs are created

  kashif

  committed
  5a36b72

  Copy full SHA for 5a36b72

• fix the docs

  kashif

  committed
  f960a98

  Copy full SHA for f960a98

• create generate_on_policy_outputs

  kashif

  committed
  f76addf

  Copy full SHA for f76addf

Commits on Sep 2, 2024

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  c6d6a64

  Copy full SHA for c6d6a64

• loss does not need labels

  kashif

  committed
  9f5841c

  Copy full SHA for 9f5841c

• use_cache is false when gradient checkpointing is True

  kashif

  committed
  c85034e

  Copy full SHA for c85034e

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  9606634

  Copy full SHA for 9606634

Commits on Sep 4, 2024

• Merge remote-tracking branch 'upstream/main' into gkd-trainer

  kashif

  committed
  ba0dbe7

  Copy full SHA for ba0dbe7

Commits on Sep 7, 2024

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  f999a9f

  Copy full SHA for f999a9f

Commits on Sep 8, 2024

• use self.assert

  kashif

  committed
  9e28c82

  Copy full SHA for 9e28c82

• fix toc

  kashif

  committed
  79f35cf

  Copy full SHA for 79f35cf

• generate_on_policy_outputs needs token_id

  kashif

  committed
  465d65f

  Copy full SHA for 465d65f

• use papers link

  kashif

  committed
  9417646

  Copy full SHA for 9417646

• teacher_model is in eval mode so no need for disabling dropout

  kashif

  committed
  046d253

  Copy full SHA for 046d253

• log completions and use_liger

  kashif

  committed
  8a7d5e4

  Copy full SHA for 8a7d5e4

• prompt from train if no eval

  kashif

  committed
  cfef12d

  Copy full SHA for cfef12d

• fix logging and add cache empty

  kashif

  committed
  3bca448

  Copy full SHA for 3bca448

• add_generation_prompt=True

  kashif

  committed
  704a833

  Copy full SHA for 704a833

Commits on Sep 9, 2024

• fix prompts

  kashif

  committed
  dedc894

  Copy full SHA for dedc894

• Update docs/source/gkd_trainer.md

  Show description for 67d7ce2
  

  kashif

  and

  qgallouedec

  authored
  67d7ce2

  Copy full SHA for 67d7ce2

• Update docs/source/gkd_trainer.md

  Show description for e668fd8
  

  kashif

  and

  qgallouedec

  authored
  e668fd8

  Copy full SHA for e668fd8

• Update docs/source/gkd_trainer.md

  Show description for 99df6c3
  

  kashif

  and

  qgallouedec

  authored
  99df6c3

  Copy full SHA for 99df6c3

• Update examples/scripts/gkd.py

  Show description for d911a5b
  

  kashif

  and

  qgallouedec

  authored
  d911a5b

  Copy full SHA for d911a5b

• minor doc changes

  qgallouedec

  committed
  6950000

  Copy full SHA for 6950000

• Merge pull request #1 from huggingface/minor-doc-changes-gkd-trainer

  Show description for 276c74e
  kashif

  authored
  276c74e

  Copy full SHA for 276c74e

• fix temp default

  kashif

  committed
  26b03d9

  Copy full SHA for 26b03d9

• Update examples/scripts/gkd.py

  Show description for de49a60
  

  kashif

  and

  lewtun

  authored
  de49a60

  Copy full SHA for de49a60

• Update examples/scripts/gkd.py

  Show description for 93df1e9
  

  kashif

  and

  lewtun

  authored
  93df1e9

  Copy full SHA for 93df1e9

• Update examples/scripts/gkd.py

  Show description for 015428e
  

  kashif

  and

  lewtun

  authored
  015428e

  Copy full SHA for 015428e

• Update examples/scripts/gkd.py

  Show description for b19c3b9
  

  kashif

  and

  lewtun

  authored
  b19c3b9

  Copy full SHA for b19c3b9

• Update examples/scripts/gkd.py

  Show description for 839ba1b
  

  kashif

  and

  lewtun

  authored
  839ba1b

  Copy full SHA for 839ba1b

• Update examples/scripts/gkd.py

  Show description for a1e47a6
  

  kashif

  and

  lewtun

  authored
  a1e47a6

  Copy full SHA for a1e47a6

• Update examples/scripts/gkd.py

  Show description for 698bd5b
  

  kashif

  and

  lewtun

  authored
  698bd5b

  Copy full SHA for 698bd5b

• Update examples/scripts/gkd.py

  Show description for 8a46719
  

  kashif

  and

  lewtun

  authored
  8a46719

  Copy full SHA for 8a46719

• Update examples/scripts/gkd.py

  Show description for ebb67d8
  

  kashif

  and

  lewtun

  authored
  ebb67d8

  Copy full SHA for ebb67d8

• Update examples/scripts/gkd.py

  Show description for 1fa75e7
  

  kashif

  and

  lewtun

  authored
  1fa75e7

  Copy full SHA for 1fa75e7

• Update examples/scripts/gkd.py

  Show description for 22d3192
  

  kashif

  and

  lewtun

  authored
  22d3192

  Copy full SHA for 22d3192

• Update examples/scripts/gkd.py

  Show description for e226453
  

  kashif

  and

  lewtun

  authored
  e226453

  Copy full SHA for e226453

• Update examples/scripts/gkd.py

  Show description for e07e499
  

  kashif

  and

  lewtun

  authored
  e07e499

  Copy full SHA for e07e499

• Update examples/scripts/gkd.py

  Show description for 293f4fe
  

  kashif

  and

  lewtun

  authored
  293f4fe

  Copy full SHA for 293f4fe

• Update examples/scripts/gkd.py

  Show description for 5475e55
  

  kashif

  and

  lewtun

  authored
  5475e55

  Copy full SHA for 5475e55

• Update examples/scripts/gkd.py

  Show description for 52d28e7
  

  kashif

  and

  lewtun

  authored
  52d28e7

  Copy full SHA for 52d28e7

• Update examples/scripts/gkd.py

  Show description for 6247387
  

  kashif

  and

  lewtun

  authored
  6247387

  Copy full SHA for 6247387

• Update examples/scripts/gkd.py

  Show description for ea2f932
  

  kashif

  and

  lewtun

  authored
  ea2f932

  Copy full SHA for ea2f932

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  c6bb85b

  Copy full SHA for c6bb85b

• Update examples/scripts/gkd.py

  Show description for 5bcfc40
  

  kashif

  and

  lewtun

  authored
  5bcfc40

  Copy full SHA for 5bcfc40

• Update examples/scripts/gkd.py

  Show description for 16e5d28
  

  kashif

  and

  lewtun

  authored
  16e5d28

  Copy full SHA for 16e5d28

• Update examples/scripts/gkd.py

  Show description for 260b6df
  

  kashif

  and

  lewtun

  authored
  260b6df

  Copy full SHA for 260b6df

• update docs

  kashif

  committed
  dd97486

  Copy full SHA for dd97486

• fix dataset format

  kashif

  committed
  24299d6

  Copy full SHA for 24299d6

• fix dataset format

  kashif

  committed
  af4f32f

  Copy full SHA for af4f32f

• no need for scores in generation

  kashif

  committed
  c5f44d8

  Copy full SHA for c5f44d8

• teacher_model_init_kwargs

  kashif

  committed
  b9cddd4

  Copy full SHA for b9cddd4

Commits on Sep 11, 2024

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  b57e350

  Copy full SHA for b57e350

• Merge branch 'main' into gkd-trainer

  lewtun

  authored
  3a13ba4

  Copy full SHA for 3a13ba4

• Update _toctree.yml

  kashif

  authored
  d00678d

  Copy full SHA for d00678d

• Update docs/source/gkd_trainer.md

  Show description for c3f31df
  

  kashif

  and

  lewtun

  authored
  c3f31df

  Copy full SHA for c3f31df

• Update tests/test_gkd_trainer.py

  Show description for a3580a6
  

  kashif

  and

  lewtun

  authored
  a3580a6

  Copy full SHA for a3580a6

• Update docs/source/gkd_trainer.md

  Show description for 57f8886
  

  kashif

  and

  lewtun

  authored
  57f8886

  Copy full SHA for 57f8886

• Update examples/scripts/gkd.py

  Show description for cf46e8a
  

  kashif

  and

  lewtun

  authored
  cf46e8a

  Copy full SHA for cf46e8a

• Update examples/scripts/gkd.py

  Show description for 5b1f9e1
  

  kashif

  and

  lewtun

  authored
  5b1f9e1

  Copy full SHA for 5b1f9e1

• Update examples/scripts/gkd.py

  Show description for 8629d4b
  

  kashif

  and

  lewtun

  authored
  8629d4b

  Copy full SHA for 8629d4b

• Update examples/scripts/gkd.py

  Show description for 25960bc
  

  kashif

  and

  lewtun

  authored
  25960bc

  Copy full SHA for 25960bc

• Update examples/scripts/gkd.py

  Show description for 14db77b
  

  kashif

  and

  lewtun

  authored
  14db77b

  Copy full SHA for 14db77b

• Update examples/scripts/gkd.py

  Show description for d8511c6
  

  kashif

  and

  lewtun

  authored
  d8511c6

  Copy full SHA for d8511c6

• Merge branch 'main' into gkd-trainer

  lewtun

  authored
  9b6339c

  Copy full SHA for 9b6339c

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  58a4dd4

  Copy full SHA for 58a4dd4

• fix

  kashif

  committed
  e8099f9

  Copy full SHA for e8099f9

• Merge branch 'main' into gkd-trainer

  kashif

  authored
  baa2429

  Copy full SHA for baa2429

• remove rich

  kashif

  committed
  13a7956

  Copy full SHA for 13a7956

• add determinstic test

  kashif

  committed
  45049dd

  Copy full SHA for 45049dd

• fix code

  kashif

  committed
  55e7638

  Copy full SHA for 55e7638

• use bigger teacher model

  kashif

  committed
  8de72e8

  Copy full SHA for 8de72e8