This is the pytorch implementation of our paper "ReActNet: Towards Precise Binary NeuralNetwork with Generalized Activation Functions", published in ECCV 2020.
In this paper, we propose to generalize the traditional Sign and PReLU functions to RSign and RPReLU, which enable explicit learning of the distribution reshape and shift at near-zero extra cost. By adding simple learnable bias, ReActNet achieves 69.4% top-1 accuracy on Imagenet dataset with both weights and activations being binary, a near ResNet-level accuracy.
If you find our code useful for your research, please consider citing:
@inproceedings{liu2020reactnet,
title={ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions},
author={Liu, Zechun and Shen, Zhiqiang and Savvides, Marios and Cheng, Kwang-Ting},
booktitle={European Conference on Computer Vision (ECCV)},
year={2020}
}
- python3, pytorch 1.4.0, torchvision 0.5.0
- Download ImageNet dataset
(1) Step1: binarizing activations
- Change directory to
./resnet/1_step1/or./mobilenet/1_step1/ - run
bash run.sh
(2) Step2: binarizing weights + activations
- Change directory to
./resnet/2_step2/or./mobilenet/2_step2/ - run
bash run.sh
| Methods | Top1-Acc | FLOPs | Trained Model |
|---|---|---|---|
| XNOR-Net | 51.2% | 1.67 x 10^8 | - |
| Bi-Real Net | 56.4% | 1.63 x 10^8 | - |
| Real-to-Binary | 65.4% | 1.83 x 10^8 | - |
| ReActNet (Bi-Real based) | 65.9% | 1.63 x 10^8 | Model-ReAct-ResNet |
| ReActNet-A | 69.5% | 0.87 x 10^8 | Model-ReAct-MobileNet |
Zechun Liu, HKUST (zliubq at connect.ust.hk)
Zhiqiang Shen, CMU (zhiqians at andrew.cmu.edu)