Skip to content

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

License

Notifications You must be signed in to change notification settings

RUkiki/channel-pruning

 
 

Repository files navigation

ICCV 2017, by Yihui He, Xiangyu Zhang and Jian Sun

Please have a look our new works on compressing deep models:

In this repository, we released code for the following models:

model Speed-up Accuracy
VGG-16 channel pruning 5x 88.1 (Top-5), 67.8 (Top-1)
VGG-16 3C1 4x 89.9 (Top-5), 70.6 (Top-1)
ResNet-50 2x 90.8 (Top-5), 72.3 (Top-1)
faster RCNN 2x 36.7 (AP@.50:.05:.95)
faster RCNN 4x 35.1 (AP@.50:.05:.95)
i2 i1
Structured simplification methods Channel pruning (d)

Citation

If you find the code useful in your research, please consider citing:

@InProceedings{He_2017_ICCV,
author = {He, Yihui and Zhang, Xiangyu and Sun, Jian},
title = {Channel Pruning for Accelerating Very Deep Neural Networks},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}
}

Contents

  1. Requirements
  2. Installation
  3. Channel Pruning and finetuning
  4. Pruned models for download
  5. Pruning faster RCNN
  6. FAQ

requirements

  1. Python3 packages you might not have: scipy, sklearn, easydict, use sudo pip3 install to install.
  2. For finetuning with 128 batch size, 4 GPUs (~11G of memory)

Installation (sufficient for the demo)

  1. Clone the repository

    # Make sure to clone with --recursive
    git clone --recursive https://github.com/yihui-he/channel-pruning.git
  2. Build my Caffe fork (which support bicubic interpolation and resizing image shorter side to 256 then crop to 224x224)

    cd caffe
    
    # If you're experienced with Caffe and have all of the requirements installed, then simply do:
    make all -j8 && make pycaffe
    # Or follow the Caffe installation instructions here:
    # http://caffe.berkeleyvision.org/installation.html
    
    # you might need to add pycaffe to PYTHONPATH, if you've already had a caffe before
  3. Download ImageNet classification dataset http://www.image-net.org/download-images

  4. Specify imagenet source path in temp/vgg.prototxt (line 12 and 36)

Channel Pruning

For fast testing, you can directly download pruned model. See next section

  1. Download the original VGG-16 model http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel
    move it to temp/vgg.caffemodel (or create a softlink instead)

  2. Start Channel Pruning

    python3 train.py -action c3 -caffe [GPU0]
    # or log it with ./run.sh python3 train.py -action c3 -caffe [GPU0]
    # replace [GPU0] with actual GPU device like 0,1 or 2
  3. Combine some factorized layers for further compression, and calculate the acceleration ratio. Replace the ImageData layer of temp/cb_3c_3C4x_mem_bn_vgg.prototxt with temp/vgg.prototxt's

    ./combine.sh | xargs ./calflop.sh
  4. Finetuning

    caffe train -solver temp/solver.prototxt -weights temp/cb_3c_vgg.caffemodel -gpu [GPU0,GPU1,GPU2,GPU3]
    # replace [GPU0,GPU1,GPU2,GPU3] with actual GPU device like 0,1,2,3
  5. Testing

    Though testing is done while finetuning, you can test anytime with:

    caffe test -model path/to/prototxt -weights path/to/caffemodel -iterations 5000 -gpu [GPU0]
    # replace [GPU0] with actual GPU device like 0,1 or 2

Pruned models (for download)

For fast testing, you can directly download pruned model from release: VGG-16 3C 4X, VGG-16 5X, ResNet-50 2X. Or follow Baidu Yun Download link

Test with:

caffe test -model channel_pruning_VGG-16_3C4x.prototxt -weights channel_pruning_VGG-16_3C4x.caffemodel -iterations 5000 -gpu [GPU0]
# replace [GPU0] with actual GPU device like 0,1 or 2

Pruning faster RCNN

For fast testing, you can directly download pruned model from release
Or you can:

  1. clone my py-faster-rcnn repo: https://github.com/yihui-he/py-faster-rcnn
  2. use the pruned models from this repo to train faster RCNN 2X, 4X, solver prototxts are in https://github.com/yihui-he/py-faster-rcnn/tree/master/models/pascal_voc

FAQ

You can find answers of some commonly asked questions in our Github wiki, or just create a new issue

About

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 99.7%
  • Shell 0.3%