Boosted-Dynamic-Networks

Haichao Yu, Haoxiang Li, Gang Hua, Gao Huang, Humphrey Shi

This repository is the official implementation for our paper Boosted Dynamic Neural Networks. In the paper, we propose a new early-exiting dynamic neural network (EDNN) architecture, where we formulate an EDNN as an additive model inspired by gradient boosting, and propose multiple training techniques to optimize the model effectively. Our experiments show it achieves superior performance on CIFAR100 and ImageNet datasets in both anytime and budgeted-batch prediction modes.

Results in Anytime Prediction Mode

MSDNet on CIFAR100	MSDNet on ImageNet	RANet on CIFAR100	RANet on ImageNet

Results in Budgeted-batch Prediction Mode

MSDNet on CIFAR100	MSDNet on ImageNet	RANet on CIFAR100	RANet on ImageNet

Usage

Please use the scripts in msdnet_scripts/ and ranet_scripts/ for model training and evaluation. For ImageNet experiments, please first download the dataset and put it into the proper folder.

Citation

@article{yu2022boostdnn,
	title        = {Boosted Dynamic Neural Networks},
	author       = {Yu, Haichao and Li, Haoxiang and Hua, Gang and Huang, Gao and Shi, Humphrey},
	year         = 2022,
	url          = {https://arxiv.org/abs/2211.16726},
	eprint       = {2211.16726},
	archiveprefix = {arXiv},
	primaryclass = {cs.LG}
}

Acknowledgments

This repository is built based on previous open-sourced efforts:

• MSDNet-PyTorch
• IMTA
• RANet-pytorch