Skip to content

Latest commit

 

History

History
201 lines (153 loc) · 9.61 KB

README.md

File metadata and controls

201 lines (153 loc) · 9.61 KB
Raw

Filter Sketch for Network Pruning (Link).

Pruning neural network model via filter sketch.

Tips

Any problem, free to contact the authors via emails: lmbxmu@stu.xmu.edu.cn or shaojieli@stu.xmu.edu.cn. Do not post issues with github as much as possible, just in case that I could not receive the emails from github thus ignore the posted issues.

Citation

If you find FilterSketch useful in your research, please consider citing:

@article{lin2020filter,
  title={Filter Sketch for Network Pruning},
  author={Lin, Mingbao and Ji, Rongrong and Li, Shaojie and Ye, Qixiang and Tian, Yonghong and Liu, Jianzhuang and Tian, Qi},
  journal={arXiv preprint arXiv:2001.08514},
  year={2020}
}

Pre-trained Models

We provide the pre-trained models used in our paper.

CIFAR-10

| ResNet56 | ResNet110 |GoogLeNet |

ImageNet

| ResNet50 |

Result Models

We provide our pruned models in the experiments, along with their training loggers and configurations.

DataSet Sketch Rate Flops
(Prune Rate）		 Params
(Prune Rate）		 Top-1 Accuracy Top-5 Accuracy Download
ResNet56 CIFAR-10 [0.6]*27 73.36M(41.5%) 0.50M(41.2%) 93.19% - Link
ResNet110 CIFAR-10 [0.9]*3+[0.4]*24+[0.3]*24+[0.9]*3 92.84M(63.3%) 0.69M(59.9%) 93.44% - Link
GoogLeNet CIFAR-10 [0.25]*9 0.59B(61.1%) 2.61M(57.6%) 94.88% - Link
ResNet50 ImageNet [0.2]*16 0.93B(77.3%) 7.18M(71.8%) 69.43% 89.23% Link
ResNet50 ImageNet [0.4]*16 1.51B(63.1%) 10.40M(59.2%) 73.04% 91.18% Link
ResNet50 ImageNet [0.6]*16 2.23B(45.5%) 14.53M(43.0%) 74.68% 92.17% Link
ResNet50 ImageNet [0.7]*16 2.64B(35.5%) 16.95M(33.5%) 75.22% 92.41% Link

Performance of FilterSketch using ResNet-56 under different compression rates.

DataSet Sketch Rate Flops
(Prune Rate）		 Params
(Prune Rate）		 Top-1 Accuracy Download
CIFAR-10 [0.1]*27 11.43M(91.0%) 0.08M(90.45%) 87.38% Link
CIFAR-10 [0.2]*27 24.54M(80.6%) 0.16M(81.0%) 90.19% Link
CIFAR-10 [0.3]*27 35.61M(71.9%) 0.25M(70.6%) 91.65% Link
CIFAR-10 [0.4]*27 48.72M(61.5%) 0.33M(61.1%) 92.00% Link
CIFAR-10 [0.5]*27 63.78M(49.6%) 0.43M(49.8%) 92.29% Link
CIFAR-10 [0.6]*27 73.36M(41.5%) 0.50M(41.2%) 93.19% Link
CIFAR-10 [0.7]*27 87.31M(31.0%) 0.59M(31.1%) 93.36% Link
CIFAR-10 [0.8]*27 98.40M(22.3%) 0.68M(20.8%) 93.40% Link
CIFAR-10 [0.9]*27 111.5M(11.9%) 0.75M(11.3%) 93.44% Link
CIFAR-10 [0.9]*3+[0.1]*10+[0.1]*10+[0.6]*4 32.47M(74.4%) 0.24M(71.8%) 91.20% Link
CIFAR-10 [0.7]*3+[0.4]*10+[0.4]*10+[0.9]*4 62.63M(50.5%) 0.48M(43.3%) 92.94% Link
CIFAR-10 [0.8]*3+[0.5]*10+[0.8]*10+[0.9]*4 88.05M(30.4%) 0.68M(20.6%) 93.65% Link

Running Code

The code has been tested using Pytorch1.3 and CUDA10.0 on Ubuntu16.04.

Filter Sketch

You can run the following code to sketch model on Cifar-10:

python sketch_cifar.py 
--data_set cifar10 
--data_path ../data/cifar10/
--sketch_model ./experiment/pretrain/resnet56.pt 
--job_dir ./experiment/resnet56/sketch/
--arch resnet 
--cfg resnet56 
--lr 0.01
--lr_decay_step 50 100
--num_epochs 150 
--gpus 0
--sketch_rate [0.6]*27
--weight_norm_method l2

You can run the following code to sketch model on Imagenet:

python sketch_imagenet.py 
--data_set imagenet 
--data_path ../data/imagenet/
--sketch_model ./experiment/pretrain/resnet50.pth 
--job_dir ./experiment/resnet50/sketch/
--arch resnet 
--cfg resnet50 
--lr 0.1
--lr_decay_step 30 60
--num_epochs 90 
--gpus 0
--sketch_rate [0.6]*16
--weight_norm_method l2

Test Our Performance

Follow the command below to verify our pruned models:

python test.py 
--data_set cifar10 
--data_path ../data/cifar10 
--arch resnet 
--cfg resnet56 
--sketch_model ./experiment/result/sketch_resnet56.pt 
--sketch_rate [0.6]*27 
--gpus 0

Get FLOPS and Params

You can use the following command to install the thop python package when you need to calculate the flops of the model:

pip install thop
python get_flops_params.py 
--data_set cifar10 
--input_image_size 32 
--arch resnet 
--cfg resnet56
--sketch_rate [0.6]*27

Remarks

The number of pruning rates required for different networks is as follows:

CIFAR-10 ImageNet
ResNet56 27 -
ResNet110 54 -
GoogLeNet 9 -
ResNet50 - 16

Other Arguments

optional arguments:
  -h, --help            show this help message and exit
  --gpus GPUS [GPUS ...]
                        Select gpu_id to use. default:[0]
  --data_set DATA_SET   Select dataset to train. default:cifar10
  --data_path DATA_PATH
                        The dictionary where the input is stored.
                        default:/home/lishaojie/data/cifar10/
  --job_dir JOB_DIR     The directory where the summaries will be stored.
                        default:./experiments
  --arch ARCH           Architecture of model. default:resnet
  --cfg CFG             Detail architecuture of model. default:resnet56
  --num_epochs NUM_EPOCHS
                        The num of epochs to train. default:150
  --train_batch_size TRAIN_BATCH_SIZE
                        Batch size for training. default:128
  --eval_batch_size EVAL_BATCH_SIZE
                        Batch size for validation. default:100
  --momentum MOMENTUM   Momentum for MomentumOptimizer. default:0.9
  --lr LR               Learning rate for train. default:1e-2
  --lr_decay_step LR_DECAY_STEP [LR_DECAY_STEP ...]
                        the iterval of learn rate. default:50, 100
  --weight_decay WEIGHT_DECAY
                        The weight decay of loss. default:5e-4
  --start_conv START_CONV
                        The index of Conv to start sketch, index starts from
                        0. default:1
  --sketch_rate SKETCH_RATE
                        The proportion of each layer reserved after sketching
                        convolution layer sketch. default:None
  --sketch_model SKETCH_MODEL
                        Path to the model wait for sketch. default:None
  --weight_norm_method WEIGHT_NORM_METHOD
                        Select the weight norm method. default:None
                        Optional:l2