-
Notifications
You must be signed in to change notification settings - Fork 1.7k
MOT 16 evaluation
-
Download this model, trained on the crowd-human-dataset, and place it under Yolov5_StrongSORT_Pytorch/yolov5/weights/
-
Run the evaluation script
python val.py --benchmark MOT16 --tracking-method strongsortThe best metrics for each release can be found here
https://github.com/mikel-brostrom/Yolov5_DeepSort_Pytorch/releases
NOTICE! This MOT16 evaluation is performed on the train split, NOT the test split, as the test ground truth is not publicly available. However, this is not an issue as the train dataset is actually never used for training. Moreover, our detector is a modest Yolov5m trained on the CrowdHuman dataset which is limiting our results.
| Method | HOTA(↑) | IDF1(↑) | MOTA(↑) | AssA(↑) | DetA(↑) | IDs(↓) | FPS(↑) |
|---|---|---|---|---|---|---|---|
| SORT | 34.0 | 39.8 | 43.1 | 31.8 | 37.0 | 4,852 | 143.3 |
| DAN | 39.3 | 49.5 | 52.4 | 36.3 | 43.1 | 8,431 | 6.3 |
| TPM | 41.5 | 52.6 | 54.2 | 40.9 | 42.5 | 1,824 | 0.8 |
| DeepMOT | 42.4 | 53.8 | 53.7 | 42.7 | 42.5 | 1,947 | 4.9 |
| Tracktor++ | 44.8 | 55.1 | 56.3 | 45.1 | 44.9 | 1,987 | 1.5 |
| TubeTK | 48.0 | 58.6 | 63.0 | 45.1 | 51.4 | 4,137 | 3.0 |
| ArTIST | 48.9 | 59.7 | 62.3 | 48.3 | 50.0 | 2,062 | 4.5 |
| MNPTrack | 49.0 | 61.7 | 58.8 | 51.1 | 47.3 | 1,185 | 6.5 |
| CenterTrack | 52.2 | 64.7 | 67.8 | 51.0 | 53.8 | 3,039 | 3.8 |
| Yolov5mStrongSORTOSNet | 53.9 | 68.8 | 60.1 | 57.5 | 51.1 | 782 | 10.2 |
| TransTrack | 54.1 | 63.5 | 75.2 | 47.9 | 61.6 | 3,603 | 59.2 |
| TransCenter | 54.5 | 62.2 | 73.2 | 49.7 | 60.1 | 4,614 | 1.0 |
| GSDT | 55.5 | 68.7 | 66.2 | 54.8 | 56.4 | 3,318 | 4.9 |
| PermaTrack | 55.5 | 68.9 | 73.8 | 53.1 | 58.5 | 3,699 | 11.9 |
| MAT | 56.0 | 69.2 | 67.1 | 57.2 | 55.1 | 1,279 | 11.5 |
| CSTrack | 59.3 | 72.6 | 74.9 | 57.9 | 61.1 | 3,567 | 15.8 |
| FairMOT | 59.3 | 72.3 | 73.7 | 58.0 | 60.9 | 3,303 | 25.9 |
| ReMOT | 59.7 | 72.0 | 77.0 | 57.1 | 62.8 | 2,853 | 1.8 |
| CrowdTrack | 60.3 | 73.6 | 75.6 | 59.3 | 61.5 | 2,544 | 140.8 |
| CorrTracker | 60.7 | 73.6 | 76.5 | 58.9 | 62.9 | 3,369 | 15.6 |
| RelationTrack | 61.0 | 74.7 | 73.8 | 61.5 | 60.6 | 1,374 | 8.5 |
| TransMOT | 61.7 | 75.1 | 76.7 | 59.9 | 63.7 | 2,346 | 1.1 |
| GRTU | 62.0 | 75.0 | 74.9 | 62.1 | 62.1 | 1,812 | 3.6 |
| MAATrack | 62.0 | 75.9 | 79.4 | 60.2 | 64.2 | 1,452 | 189.1 |
| ByteTrack | 63.1 | 77.3 | 80.3 | 62.0 | 64.5 | 2,196 | 29.6 |
| DeepSORT | 61.2 | 74.5 | 78.0 | 59.7 | 63.1 | 1,821 | 13.8 |
| StrongSORT | 63.5 | 78.5 | 78.3 | 63.7 | 63.6 | 1,446 | 7.5 |
We achieve state of the art results in IDs (Number of identity switches) by a large margin. By hovering over the metric in this table you can read about them.
Thanks to zengjie617789 for helping out with this evaluation!