Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #71

Merged
merged 28 commits into from
Jul 21, 2022
Merged

Develop #71

merged 28 commits into from
Jul 21, 2022

Conversation

BAAI-OpenPlatform
Copy link
Collaborator

No description provided.

920232796 and others added 23 commits July 1, 2022 15:24
* clean codes 
Co-authored-by: Zac Liu <liuguang@baai.ac.cn>
* fix bert tokenizer issue
* updated t5, opt and roberta tokenizers
* fixed doc 404 error
Signed-off-by: ZhaodongYan1 <yzdrogeryan@gmail.com>
* autoloader for opt
* opt-66b inference
* Update train.py
* Load data from example dir
* add readme of multi GPU inference

Co-authored-by: Zac Liu <liuguang@baai.ac.cn>
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
Signed-off-by: Anhforth <yanzhaodong2021@163.com>
remove unused glob
remove data statis
Signed-off-by: zhaohu xing <920232796@qq.com>
Signed-off-by: zhaohu xing <920232796@qq.com>
Signed-off-by: zhaohu xing <920232796@qq.com>
* add vit and examples

* vit and examples

* Update base_model.py

remove unused glob

* Update vit.py

remove data statis

* modify readme.md

Signed-off-by: zhaohu xing <920232796@qq.com>

* modify readme.md

Signed-off-by: zhaohu xing <920232796@qq.com>

* delete annotating code

Signed-off-by: zhaohu xing <920232796@qq.com>

Co-authored-by: Zac Liu <liuguang@baai.ac.cn>
merge flagi  into internal develop
Signed-off-by: zhaohu xing <920232796@qq.com>
add env_trainer for batch script running
fix typo in flagai introduction.

model = loader.get_model()

model.load_state_dict(torch.load(os.path.join(model_save_dir, "38000", "pytorch_model.bin"), map_location=device)["module"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change to from_pretrain to load the model

x = self.patch_embed(x)
if self.cls_token is not None:
x = torch.cat((self.cls_token.expand(x.shape[0], -1, -1), x), dim=1)
x = self.pos_drop(x + self.pos_embed)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add checkpoint_activations

920232796 and others added 3 commits July 21, 2022 18:09
Signed-off-by: zhaohu xing <920232796@qq.com>
Signed-off-by: zhaohu xing <920232796@qq.com>
…vations

add  checkpointing activations for VIT when env_type = pytorch/deepspeed
@marscrazy marscrazy merged commit 2ee645e into master Jul 21, 2022
@marscrazy marscrazy linked an issue Jul 21, 2022 that may be closed by this pull request
@BAAI-OpenPlatform BAAI-OpenPlatform deleted the develop branch August 25, 2022 09:04
BAAI-OpenPlatform added a commit that referenced this pull request Nov 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Typo found in the first sentence of readme file
4 participants