Centered Weight Normalization

This project is the Torch implementation of the paper: Centered Weight Normalization in Accelerating Training of Deep Neural Networks ( ICCV 2017).

• bibtex:

@INPROCEEDINGS{Huang2017ICCV,
    author = {Lei Huang and Xianglong Liu and Yang Liu and  Bo Lang and Dacheng Tao},
    title = {Centered Weight Normalization  in Accelerating Training of Deep Neural Networks},
    booktitle = {ICCV},
    year = {2017}}

Updates

• Add the ' pytorch module' of CWN. 2019-12-07.

Requirements and Dependency

• Install Torch with CUDA GPU
• Install cudnn v5
• Install dependent lua packages optnet by run: luarocks install optnet

Experiments in the paper

1. MLP architecture over SVHN dataset

• Dataset prepraration: We can get the preprocessed SVHN dataset for MLP architecture by running:

  cd dataset
   th preProcess_div256.lua

Note that this script is based on the Torch script for SVHN

• Execute:

 th exp_MLP.lua 

• To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:

 bash 1_execute_MLP_svhn.sh  
 bash 1_execute_MLP_svhn_adam.sh  

2. VGG-A architecture over Cifar-10 dataset

• Dataset preparations: the dataset is based on the preprocessed script on: https://github.com/szagoruyko/cifar.torch, and you should put the data file in the directory: './dataset/cifar_provider.t7'

• Execute:

 th exp_vggA.lua –dataPath './dataset/cifar_provider.t7'

• To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:

 bash   2_execute_Conv_CIFAR10_vggA.sh

3. GoogLeNet architecture over Cifar datasets

• Dataset preparations: The dataset is based on whitened CIFAR datasets.
• Execute:

th exp_GoogleNet_dataWhitening.lua –dataPath './dataset/cifar100_whitened.t7'

• To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:

 3_execute_Conv_CIFAR100_GoogLeNet.sh 

The GoogLeNet model is based on the project on: https://github.com/soumith/imagenet-multiGPU.torch

4. Residual network architecture over Cifar datasets

• Dataset preparations: The dataset is based on original CIFAR datasets, and the data file should be put in the directory: ./dataset/cifar_original.t7.
• Execute:

th exp_res_dataNorm.lua –dataPath './dataset/cifar10_original.t7'

• To reproduce the experimental results, you can run the script below, which include all the information of experimental configurations:

4_execute_Conv_CIFAR10_resnet.sh

The normlization of Cifar dataset is in the script th exp_res_dataNorm.lua. The residual network model and respective script are based on facebook ResNet.

5. GoogLeNet over ImageNet

This experiment is based on the project at: https://github.com/soumith/imagenet-multiGPU.torch.
The proposed model are in: './models/imagenet/'

Contact

huanglei@nlsde.buaa.edu.cn, Any discussions and suggestions are welcome!