Accuracy of the pruned model

[zhoudaquan]

Hi,

I have followed the code here and run the sparse training code as below:

python main.py --arch vgg11_bn --s 0.00001 --save [PATH TO SAVE RESULTS] [IMAGENET]

After the training, the accuracy is 71.4% which is fine. However, The pruning results is almost 0 with the 0.5 pruning ratio. As I decrease the pruning ratio to be 0.2, the top 1 accuracy increase to 15% which also far below the expectation. Could you please advice this is normal or there could be something wrong?

I would like a one time pruning and do not want to prune iterative.

Thanks for your reply.

Best regards,

[Eric-mingjie]

Are you training on ImageNet? You could finetune the model after pruning.

[zhoudaquan]

Are you training on ImageNet? You could finetune the model after pruning.

Hi,

Thanks for the fast response.

Yeah... I am training on ImageNet dataset. I just plot the bn scaling factor from the sparsely trained model and I found the resulting values is not sparse actually...

Is this normal or there could be something wrong?

Thanks for the advice.

[zhoudaquan]

Are you training on ImageNet? You could finetune the model after pruning.

yeah... I could fine tune the model. I am new to model pruning and I have no idea regarding the performance. Is this normal practice? Is the performance dropping too much for the first round pruning?

[Eric-mingjie]

I am not exactly sure of the performance of the pruned model. One reason for the low accuracy could be that the sparsity loss factor 0.00001 is small. You can try increasing it.

[zhoudaquan]

I am not exactly sure of the performance of the pruned model. One reason for the low accuracy could be that the sparsity loss factor 0.00001 is small. You can try increasing it.

Thanks for the reply.