Code

BMTrain New Version Release v1.0.0#182

Code

Merged

MayDomine merged 62 commits intomainOpenBMB/BMTrain:mainfrom devOpenBMB/BMTrain:devCopy head branch name to clipboardFeb 26, 2024

+3,492-1,772

Commits

Commits on Jul 21, 2023

• remove inappropriate import in __init__.py

  Achazwl

  authored
  37c2edc

  Copy full SHA for 37c2edc

Commits on Aug 17, 2023

• Merge pull request #126 from OpenBMB/clean_init

  Show description for c47d3c8
  Achazwl

  authored
  c47d3c8

  Copy full SHA for c47d3c8

Commits on Aug 18, 2023

• FIX tensor slice in gather

  JerryYin777

  authored
  15de53c

  Copy full SHA for 15de53c

• Update synchronize.py

  JerryYin777

  authored
  6553504

  Copy full SHA for 6553504

• FIX Error : tensor slice in gather()

  JerryYin777

  authored
  0c6da0b

  Copy full SHA for 0c6da0b

• Update synchronize.py

  JerryYin777

  authored
  7d4a570

  Copy full SHA for 7d4a570

• Create test_synchronize.py

  JerryYin777

  authored
  245dcaa

  Copy full SHA for 245dcaa

• Update test_synchronize.py

  JerryYin777

  authored
  7484cfb

  Copy full SHA for 7484cfb

• add test_synchronize

  JerryYin777

  authored
  e0f8c30

  Copy full SHA for e0f8c30

Commits on Aug 19, 2023

• Merge branch 'main' of github.com:OpenBMB/BMTrain into dev

  Achazwl

  committed
  75aa1a8

  Copy full SHA for 75aa1a8

• add ReduceScatter communication op

  MayDomine

  committed
  99e0706

  Copy full SHA for 99e0706

Commits on Aug 21, 2023

• FIX: unittest and backward for reduce scatter

  MayDomine

  committed
  6213886

  Copy full SHA for 6213886

• Refactor ZeRO, checkpoint and pipeline code (#128)

  Show description for 74700e4
  

  zkh2016

  and

  zhangkaihuo

  authored
  74700e4

  Copy full SHA for 74700e4

• prod backward raise error

  MayDomine

  committed
  b0a0865

  Copy full SHA for b0a0865

• fix inspect_model where param is None (#151)

  zkh2016

  authored
  3f11744

  Copy full SHA for 3f11744

Commits on Aug 22, 2023

• fix is_first_layer (#152)

  zkh2016

  authored
  dc7284d

  Copy full SHA for dc7284d

Commits on Aug 25, 2023

• mv zero_level to CheckpointBlock (#154)

  zkh2016

  authored
  7d62a18

  Copy full SHA for 7d62a18

Commits on Aug 28, 2023

• FIX：the number of GPU of synchronize

  JerryYin777

  authored
  286297e

  Copy full SHA for 286297e

• add judgement through assert

  JerryYin777

  authored
  69ea264

  Copy full SHA for 69ea264

• Add

  JerryYin777

  authored
  17973a2

  Copy full SHA for 17973a2

• Merge pull request #146 from JerryYin777/dev

  Show description for 9b944f8
  Achazwl

  authored
  9b944f8

  Copy full SHA for 9b944f8

• FIX: Revise the document path

  JerryYin777

  authored
  d435097

  Copy full SHA for d435097

• Fix middle hidden (#155)

  zkh2016

  authored
  abc7b90

  Copy full SHA for abc7b90

Commits on Aug 29, 2023

• Tensor Parallel (#153)

  zkh2016

  authored
  df43d6d

  Copy full SHA for df43d6d

• Add Bf16 Support (#136)

  Achazwl

  authored
  38461bc

  Copy full SHA for 38461bc

Commits on Aug 30, 2023

• Rename class and files (#157)

  zkh2016

  authored
  a7fb078

  Copy full SHA for a7fb078

• Add zero_context.py (#160)

  zkh2016

  authored
  90492fd

  Copy full SHA for 90492fd

Commits on Aug 31, 2023

• FIX error of multi-gpus of test_synchronize.py

  JerryYin777

  authored
  9bbd279

  Copy full SHA for 9bbd279

• Refactor communicate groups and Block (#159)

  zkh2016

  authored
  f256db6

  Copy full SHA for f256db6

• Update test_synchronize.py (#161)

  JerryYin777

  authored
  0ba1e3a

  Copy full SHA for 0ba1e3a

• Fix cross (#162)

  zkh2016

  authored
  985a3c3

  Copy full SHA for 985a3c3

Commits on Sep 4, 2023

• Fix block wrapper (#166)

  zkh2016

  authored
  206e11d

  Copy full SHA for 206e11d

Commits on Sep 6, 2023

• Fix async send in pipe mode (#167)

  zkh2016

  authored
  696eb5c

  Copy full SHA for 696eb5c

• Faster parallel linear (#165)

  zkh2016

  authored
  535279e

  Copy full SHA for 535279e

Commits on Sep 14, 2023

• Merge branch 'dev' into main

  MayDomine

  authored
  227fec0

  Copy full SHA for 227fec0

• Merge pull request #147 from MayDomine/main

  Show description for 2f49c99
  MayDomine

  authored
  2f49c99

  Copy full SHA for 2f49c99

Commits on Sep 16, 2023

• fix async row linear not support split_input (#168)

  Achazwl

  authored
  95417b5

  Copy full SHA for 95417b5

Commits on Sep 27, 2023

• add _save_to_infer_model (#170)

  zkh2016

  authored
  25e3671

  Copy full SHA for 25e3671

• Async save state_dict to file (#171)

  zkh2016

  authored
  290386a

  Copy full SHA for 290386a

Commits on Oct 8, 2023

• init tp comm when tp_size>1

  

  zhangkaihuo

  committed
  cf6ad55

  Copy full SHA for cf6ad55

• fix tp

  

  zhangkaihuo

  committed
  abcd7c8

  Copy full SHA for abcd7c8

Commits on Nov 10, 2023

• fix adam bf16 load changed to fp16 (#175)

  Achazwl

  authored
  9dc7811

  Copy full SHA for 9dc7811

Commits on Nov 14, 2023

• Merge pull request #173 from zkh2016/fix_tp

  Show description for 00caadb
  Achazwl

  authored
  00caadb

  Copy full SHA for 00caadb

• Merge pull request #172 from zkh2016/control_tp_comm

  Show description for 31340c5
  Achazwl

  authored
  31340c5

  Copy full SHA for 31340c5

Commits on Nov 30, 2023

• Fix parallel_for when grain_size > 0 (#179)

  Show description for ed90d1f
  Achazwl

  authored
  ed90d1f

  Copy full SHA for ed90d1f

Commits on Dec 1, 2023

• fix test_training

  MayDomine

  committed
  76aeeff

  Copy full SHA for 76aeeff

• tp cross entropy (#180)

  Achazwl

  authored
  5aeba09

  Copy full SHA for 5aeba09

Commits on Dec 27, 2023

• Update optim_manager.py (#181)

  Show description for 3c80ce4
  Achazwl

  authored
  3c80ce4

  Copy full SHA for 3c80ce4

• Feat optim manager state (#176)

  Show description for 6abcf77
  Achazwl

  authored
  6abcf77

  Copy full SHA for 6abcf77

Commits on Feb 20, 2024

• FIX: allgather_object stuck

  MayDomine

  committed
  b7d26e2

  Copy full SHA for b7d26e2

• add test for allgather object

  MayDomine

  committed
  3b7b6b9

  Copy full SHA for 3b7b6b9

• Merge pull request #183 from MayDomine/dev

  Show description for e3b689c
  MayDomine

  authored
  e3b689c

  Copy full SHA for e3b689c

• fix projection interface

  MayDomine

  committed
  281860a

  Copy full SHA for 281860a

Commits on Feb 22, 2024

• Optimizer load gathered state and record delta feature are supported now

  MayDomine

  committed
  d933ee9

  Copy full SHA for d933ee9

Commits on Feb 23, 2024

• fix adam offload return_delta and formatting code

  MayDomine

  committed
  8cbb576

  Copy full SHA for 8cbb576

• workflow test

  MayDomine

  committed
  fef8a7a

  Copy full SHA for fef8a7a

• update the action config yaml

  MayDomine

  committed
  75b84ac

  Copy full SHA for 75b84ac

• add trigger for pr synchronize

  MayDomine

  committed
  341caa0

  Copy full SHA for 341caa0

• fix c++ include

  MayDomine

  committed
  119e5cf

  Copy full SHA for 119e5cf

• Merge pull request #184 from OpenBMB/load_opt

  Show description for 0def29f
  MayDomine

  authored
  0def29f

  Copy full SHA for 0def29f

• Vocal parallel Embedding impl and make example work when tp_size > 1

  MayDomine

  committed
  f915f94

  Copy full SHA for f915f94

Commits on Feb 26, 2024

• Merge pull request #186 from OpenBMB/vpe

  Show description for 5713d76
  MayDomine

  authored
  5713d76

  Copy full SHA for 5713d76