Model pruning support

[lucasjinreal]

The yolov5l (s and m were same) now have a TensorRT accelerated version, It runs about 25 fps within 35 ms on GTX1080 Ti in a 1280x768 full resolution. With 800x800 or smaller input and fp16 would much more faster.

So, here it is, as far as I know, many models have many redundant weights. If yolov5 can have pruning support, it would help us found a model both good at accuracy and speed, hopefully it can cut down the whole model size for 50% without huge harm of accuracy, the speed would also massively accelerated.

[sithu31296]

@jinfagang That would be great.
May I know how you did to get TensorRT version of yolov5l?

[lucasjinreal]

@sithu31296 It need customize onnx export and written some cuda postprocess to do so. Hopefully I will post some detection result and conversion script in the next few weeks.

[sithu31296]

@jinfagang Thank you. Hope to see your results and script.

[Lornatang]

@jinfagang @sithu31296 Can refer to https://github.com/Lam1360/YOLOv3-model-pruning

[glenn-jocher]

Also see the pytorch pruning tutorial. It seems pretty simple. I'd encourage you guys to try this and submit a PR if you get it working, perhaps by writing a new file prune.py that might be called similarly to export_onnx.py, with arguments for a model etc.
https://pytorch.org/tutorials/intermediate/pruning_tutorial.html

[Lornatang]

@glenn-jocher Yes, it can be trained on your model first, and then sparse training after training is completed, and you will eventually get similar accuracy to your model, but the model size will be reduced by three to four times.

[lucasjinreal]

@glenn-jocher I'd like to have some expriments on this issue, updates will record here. @sithu31296 tensorrt yolov5 had done:

[glenn-jocher]

@jinfagang that looks good, but I think there may be some +y shift in your boxes occuring?

[lucasjinreal]

@glenn-jocher Haha, you are really have a sharp eye, am also notice this. I think it casued by int float conversion accuracy miss. will look into it.

[glenn-jocher]

It could be in the labels also. The model will output any box style it learns during training.

[glenn-jocher]

Just tried pruning now and ran into errors. Seems like the pytorch pruning code may have a bug:
pytorch/tutorials#1054

[glenn-jocher]

@jinfagang I got a workaround working and wrote a simple tutorial at https://docs.ultralytics.com/yolov5/tutorials/model_pruning_and_sparsity. Pruning seems to work fine, though it reduces mAP correspondingly with your level of pruning. Low pruning (<50%) seems to show similar qualitative results to using the full model. I did not see any speed improvements in pytorch with a P100.

[lucasjinreal]

@glenn-jocher Brilliant! Speed doesn't improvement maybe it is already fast enough, in some lower compute level card such as GTX1080TI might have visiable improvement, does the model size decreased much?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.