transformer_framework

framework for plug and play of various transformers (vision and nlp) with FSDP

Name		Name	Last commit message	Last commit date
Latest commit History 235 Commits
.vscode		.vscode
comm_tracking_files		comm_tracking_files
config		config
dataset_classes		dataset_classes
datasets_grammar		datasets_grammar
environment		environment
metric_logging		metric_logging
mlp		mlp
model_checkpointing		model_checkpointing
models		models
optimizers		optimizers
patch_embeddings		patch_embeddings
performance		performance
policies		policies
positional_embedding		positional_embedding
pytorch_mods		pytorch_mods
weight_init		weight_init
.gitignore		.gitignore
8gpu_631M_trunc_llama		8gpu_631M_trunc_llama
LICENSE		LICENSE
README.md		README.md
adanip_exp.py		adanip_exp.py
beans_mqa.txt		beans_mqa.txt
check_bnb_install.py		check_bnb_install.py
collect_env.py		collect_env.py
collect_env.py.1		collect_env.py.1
food_1B_mqa_bs40.txt		food_1B_mqa_bs40.txt
food_1B_seqqq_bs40.txt		food_1B_seqqq_bs40.txt
gc.py		gc.py
main_training.py		main_training.py
requirements.txt		requirements.txt
run_simple_training.sh		run_simple_training.sh
run_training.sh		run_training.sh
run_vitsmart.sh		run_vitsmart.sh
sharding_groups_helper.py		sharding_groups_helper.py
simple_training.py		simple_training.py
tflops_counter.py		tflops_counter.py
trainer.slurm		trainer.slurm

Provide feedback