Haichao Yu, Haoxiang Li, Gang Hua, Gao Huang, Humphrey Shi

This repository is the official implementation for our paper Boosted Dynamic Neural Networks. In the paper, we propose a new early-exiting dynamic neural network (EDNN) architecture, where we formulate an EDNN as an additive model inspired by gradient boosting, and propose multiple training techniques to optimize the model effectively. Our experiments show it achieves superior performance on CIFAR100 and ImageNet datasets in both anytime and budgeted-batch prediction modes.

MSDNet on CIFAR100	MSDNet on ImageNet	RANet on CIFAR100	RANet on ImageNet

MSDNet on CIFAR100	MSDNet on ImageNet	RANet on CIFAR100	RANet on ImageNet

Please use the scripts in msdnet_scripts/ and ranet_scripts/ for model training and evaluation. For ImageNet experiments, please first download the dataset and put it into the proper folder.

@article{yu2022boostdnn,
	title        = {Boosted Dynamic Neural Networks},
	author       = {Yu, Haichao and Li, Haoxiang and Hua, Gang and Huang, Gao and Shi, Humphrey},
	year         = 2022,
	url          = {https://arxiv.org/abs/2211.16726},
	eprint       = {2211.16726},
	archiveprefix = {arXiv},
	primaryclass = {cs.LG}
}

This repository is built based on previous open-sourced efforts:

• MSDNet-PyTorch
• IMTA
• RANet-pytorch