DIANet: Dense-and-Implicit-Attention-Network

By Zhongzhan Huang, Senwei Liang, Mingfu Liang and Haizhao Yang.

This repo is the official implementation of "DIANet: Dense-and-Implicit Attention Network" [paper] on CIFAR-100 dataset. Our paper has been accepted for POSTER presentation at the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20). You can also check with the AAAI proceeding version.

Introduction

DIANet[paper] provides a universal framework that recurrently fuses the information from preceding layers to enhance the attention modeling at each layer. The existing network backbone can embed DIA unit readily by sharing the DIA unit with all the layers. In the implementation, an LSTM-based DIA unit is provided.

Install

• Install PyTorch
• Clone the DIANet repository

  git clone https://github.com/gbup-group/DIANet.git
  

Usage

python run_code.py

Results

	original	DIANet(r=4)
ResNet164	73.43	76.67
PreResNet164	76.53	78.20
WRN52-4	79.75	80.99
ResNext101,8x32	81.18	82.46

Notes:

• After five times experiments, we show the mean performance in the table above.
• r denote reduction ratio in DIA unit.
• For more implementation details and parameter settings check the code or the appendix of our paper.
• Testing on 2*GPU(P100)

Citing DIANet

@inproceedings{huang2020dianet,
  title={DIANet: Dense-and-Implicit Attention Network.},
  author={Huang, Zhongzhan and Liang, Senwei and Liang, Mingfu and Yang, Haizhao},
  booktitle={AAAI},
  pages={4206--4214},
  year={2020}
}

Acknowledgments

Many thanks to bearpaw for his simple and clean framework for image classification task(PyTorch).