Code

Fix gpt trainer test#6915

Code

Merged

hsiehjackson merged 156 commits intomainNVIDIA/NeMo:mainfrom fix-gpt-trainer-testNVIDIA/NeMo:fix-gpt-trainer-testCopy head branch name to clipboardAug 10, 2023

+362-257

Commits

Commits on Jun 24, 2023

• Add trainer.test() for GPT

  Show description for 5f0d53b
  hsiehjackson

  committed
  5f0d53b

• Remove unused part

  Show description for e6865c8
  hsiehjackson

  committed
  e6865c8

Commits on Jun 26, 2023

• Add trainer.test() for GPT

  Show description for de3cc19
  hsiehjackson

  committed
  de3cc19

• Remove unused part

  Show description for 87ce549
  hsiehjackson

  committed
  87ce549

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 74c6fd8
  

  pre-commit-ci[bot]

  authored and

  hsiehjackson

  committed
  74c6fd8

Commits on Jun 27, 2023

• Fix training part

  Show description for 98da36e
  hsiehjackson

  committed
  98da36e

• Fix conflict

  Show description for 9656fa9
  hsiehjackson

  committed
  9656fa9

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 7c240b3
  pre-commit-ci[bot]

  committed
  7c240b3

• Fix config

  Show description for c22cfd6
  hsiehjackson

  committed
  c22cfd6

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  83ff763

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  afa9030

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  ae0f1f0

Commits on Jun 28, 2023

• Fix references and add CI

  Show description for a4588f4
  hsiehjackson

  committed
  a4588f4

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  689f96c

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 6d7c857
  pre-commit-ci[bot]

  committed
  6d7c857

• Fix config error

  Show description for 3bbc4e7
  hsiehjackson

  committed
  3bbc4e7

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  7539545

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for bad78a6
  pre-commit-ci[bot]

  committed
  bad78a6

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  f2405bc

• Fix dataset

  Show description for d74f8d9
  hsiehjackson

  committed
  d74f8d9

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  cadfaca

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 6c4950c
  pre-commit-ci[bot]

  committed
  6c4950c

• Add metadata

  Show description for 2f6d963
  hsiehjackson

  committed
  2f6d963

• Fix conflict

  Show description for 9b00e69
  hsiehjackson

  committed
  9b00e69

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for ea6968b
  pre-commit-ci[bot]

  committed
  ea6968b

Commits on Jun 29, 2023

• Fix config

  Show description for e96f4cd
  hsiehjackson

  committed
  e96f4cd

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  a347456

• Merge branch 'main' into fix-gpt-trainer-test

  Show description for c88f17f
  hsiehjackson

  authored
  c88f17f

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  b8616f4

• Fix empty batch

  Show description for 3f7074d
  hsiehjackson

  committed
  3f7074d

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  b6e0572

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 9292781
  pre-commit-ci[bot]

  committed
  9292781

• Fix config

  Show description for 63f05cc
  hsiehjackson

  committed
  63f05cc

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  4e91a40

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 20173e6
  pre-commit-ci[bot]

  committed
  20173e6

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  db27d89

Commits on Jun 30, 2023

• Fix config

  Show description for 855dbec
  hsiehjackson

  committed
  855dbec

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  6f36a52

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  546a5d6

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  2de6e3f

• Fix max seq length

  Show description for 7daf79e
  hsiehjackson

  committed
  7daf79e

• Fix dataset

  Show description for 0c3da74
  hsiehjackson

  committed
  0c3da74

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for fca0024
  pre-commit-ci[bot]

  committed
  fca0024

Commits on Jul 1, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  6735077

Commits on Jul 2, 2023

• Fix dataset

  Show description for e33d255
  hsiehjackson

  committed
  e33d255

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  da33fc8

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for f383adb
  pre-commit-ci[bot]

  committed
  f383adb

Commits on Jul 5, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  711b122

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  b3d01bd

Commits on Jul 6, 2023

• Add token f1

  Show description for aa756c5
  hsiehjackson

  committed
  aa756c5

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  caebbd5

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 2a54be3
  pre-commit-ci[bot]

  committed
  2a54be3

• Add FA in sft

  Show description for 7099fc4
  hsiehjackson

  committed
  7099fc4

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  f9396f5

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 1b43a95
  pre-commit-ci[bot]

  committed
  1b43a95

• Add inference config

  Show description for 2d6109b
  hsiehjackson

  committed
  2d6109b

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  5057605

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 463e9a6
  pre-commit-ci[bot]

  committed
  463e9a6

Commits on Jul 7, 2023

• Fix bug

  Show description for c4bad4b
  hsiehjackson

  committed
  c4bad4b

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  5e5cf2a

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  61bf69a

• Fix pad

  Show description for 734b5df
  hsiehjackson

  committed
  734b5df

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  5bba479

Commits on Jul 8, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  179039a

• Fix num batch

  Show description for c1bf6e3
  hsiehjackson

  committed
  c1bf6e3

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  516f942

Commits on Jul 10, 2023

• Add query_key

  Show description for e456359
  hsiehjackson

  committed
  e456359

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 3b7a758
  pre-commit-ci[bot]

  committed
  3b7a758

• Remove pdb

  Show description for 4a62efb
  hsiehjackson

  committed
  4a62efb

• Fix conflict

  Show description for fc37a11
  hsiehjackson

  committed
  fc37a11

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  e7aba8e

Commits on Jul 11, 2023

• Fix write json

  Show description for 2372ec5
  hsiehjackson

  committed
  2372ec5

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  f305e1f

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 54ed4da
  pre-commit-ci[bot]

  committed
  54ed4da

• Fix dataset bug and refactor

  Show description for aede844
  hsiehjackson

  committed
  aede844

• Fix conflict

  Show description for e844073
  hsiehjackson

  committed
  e844073

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 9d6aaef
  pre-commit-ci[bot]

  committed
  9d6aaef

• Add logging for prediction

  Show description for 3bca274
  hsiehjackson

  committed
  3bca274

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  b01949e

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  b7a82c6

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  01b0465

• Fix retrain

  Show description for 1fb3048
  hsiehjackson

  committed
  1fb3048

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  c7c936d

Commits on Jul 12, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  Show description for 2323131
  hsiehjackson

  authored
  2323131

• Add query_key in config

  Show description for c9329c7
  hsiehjackson

  committed
  c9329c7

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  8fd391e

Commits on Jul 13, 2023

• Fix bug

  Show description for 4dd4840
  hsiehjackson

  committed
  4dd4840

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 5413add
  pre-commit-ci[bot]

  committed
  5413add

• Fix config

  Show description for 6557e07
  hsiehjackson

  committed
  6557e07

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  3dda9ef

Commits on Jul 14, 2023

• Fix bug

  Show description for be7bb04
  hsiehjackson

  committed
  be7bb04

• Merge branch 'main' into fix-gpt-trainer-test

  Show description for cbd68dd
  hsiehjackson

  authored
  cbd68dd

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 9cd83ce
  pre-commit-ci[bot]

  committed
  9cd83ce

Commits on Jul 17, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  6715fd0

Commits on Jul 18, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  ad22848

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  b83a7bb

Commits on Jul 19, 2023

• Add inference config

  Show description for 9adcd83
  hsiehjackson

  committed
  9adcd83

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  986fc2e

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for de81d72
  pre-commit-ci[bot]

  committed
  de81d72

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  4b40783

Commits on Jul 20, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  2abb44e

Commits on Jul 21, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  7b561cd

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  c73d0bb

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  c85e135

Commits on Jul 26, 2023

• Fix bug

  Show description for c33f7b4
  hsiehjackson

  committed
  c33f7b4

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  1ec15bc

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 3315ed2
  pre-commit-ci[bot]

  committed
  3315ed2

• Fix mask

  Show description for b7d114d
  hsiehjackson

  committed
  b7d114d

• Fix mask

  Show description for 23af796
  hsiehjackson

  committed
  23af796

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for a9fba67
  pre-commit-ci[bot]

  committed
  a9fba67

• Fix mask

  Show description for 14a09f1
  hsiehjackson

  committed
  14a09f1

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  0b019a1

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  f80b651

Commits on Jul 28, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  febfc0f

Commits on Jul 31, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  2ba5aed

• Split PR

  Show description for bc28be9
  hsiehjackson

  committed
  bc28be9

• Undo commit

  Show description for 5c61dfb
  hsiehjackson

  committed
  5c61dfb

• Add query_key to doc_string

  Show description for 35f2613
  hsiehjackson

  committed
  35f2613

• Adjust yzhang123 comments

  Show description for 806b88e
  hsiehjackson

  committed
  806b88e

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 3fade76
  pre-commit-ci[bot]

  committed
  3fade76

• Fix error and follow comments

  Show description for 818a679
  hsiehjackson

  committed
  818a679

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  12bbae1

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for ba2548d
  pre-commit-ci[bot]

  committed
  ba2548d

Commits on Aug 1, 2023

• Remove query key

  Show description for 84027bf
  hsiehjackson

  committed
  84027bf

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  535bbcd

• Remove logic and query

  Show description for 44bc444
  hsiehjackson

  committed
  44bc444

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for bd2aa94
  pre-commit-ci[bot]

  committed
  bd2aa94

• Remove query from model

  Show description for 650d5ea
  hsiehjackson

  committed
  650d5ea

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  dfcec5b

Commits on Aug 2, 2023

• Remove query_key

  Show description for 4f82530
  hsiehjackson

  committed
  4f82530

• Fix error

  Show description for 64f9453
  hsiehjackson

  committed
  64f9453

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for a14f79d
  pre-commit-ci[bot]

  committed
  a14f79d

• Fix pdb

  Show description for 8f9598c
  hsiehjackson

  committed
  8f9598c

• Fix conflict

  Show description for 0125c58
  hsiehjackson

  committed
  0125c58

• Add default tokens_to_generate

  Show description for 727c750
  hsiehjackson

  committed
  727c750

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  ea3710b

Commits on Aug 4, 2023

• Revert prompt truncate re-prompt

  Show description for dfac0f0
  hsiehjackson

  committed
  dfac0f0

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  0ab4eeb

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for df4d4c1
  pre-commit-ci[bot]

  committed
  df4d4c1

• Revert

  Show description for f457c3f
  hsiehjackson

  committed
  f457c3f

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  1007152

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 407aae3
  pre-commit-ci[bot]

  committed
  407aae3

• skip generation with metric loss

  Show description for 4c712ae
  hsiehjackson

  committed
  4c712ae

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  b99f522

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 2b59de8
  pre-commit-ci[bot]

  committed
  2b59de8

• Revert

  Show description for ee74e54
  hsiehjackson

  committed
  ee74e54

• Fix conflict

  Show description for 9b9626b
  hsiehjackson

  committed
  9b9626b

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 67ec1b2
  pre-commit-ci[bot]

  committed
  67ec1b2

Commits on Aug 7, 2023

• Merge branch 'main' into fix-gpt-trainer-test

  Show description for 82a940c
  hsiehjackson

  authored
  82a940c

• Merge branch 'main' into fix-gpt-trainer-test

  hsiehjackson

  authored
  c51754e

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for ac1241f
  pre-commit-ci[bot]

  committed
  ac1241f

• Fix bug

  Show description for 29f775c
  hsiehjackson

  committed
  29f775c

Commits on Aug 10, 2023

• support GPTSFTChatDataset

  Show description for cf2f2b9
  hsiehjackson

  committed
  cf2f2b9

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 6764801
  pre-commit-ci[bot]

  committed
  6764801

• Add comment

  Show description for 9f936c0
  hsiehjackson

  committed
  9f936c0

• Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo into fix-gpt-trainer-test

  hsiehjackson

  committed
  3d3fccc