Skip to content

Latest commit

 

History

History
153 lines (137 loc) · 14.7 KB

performance.md

File metadata and controls

153 lines (137 loc) · 14.7 KB

Below two tables show the number of keypoints detected on the image for each detector and the time it took to extract those keypoints.

Detector / Total Keypoints [ms]

Detector / Keypoints 0 1 2 3 4 5 6 7 8 9
SHITOMASI 1370 1301 1361 1358 1333 1284 1322 1366 1389 1339
HARRIS 420 502 516 524 523 511 505 510 529 520
FAST 5606 4952 4863 4840 4856 4899 4870 4868 4996 4997
BRISK 2757 2777 2741 2735 2757 2695 2715 2628 2639 2672
ORB 500 500 500 500 500 500 500 500 500 500
AKAZE 1351 1327 1311 1351 1360 1347 1363 1331 1358 1331
SIFT 1438 1371 1380 1335 1305 1369 1396 1382 1463 1422

Detector / Keypoint Extraction Time [ms]

Detector / Extraction Time [ms] 0 1 2 3 4 5 6 7 8 9
SHITOMASI 22.6 6.5 15.1 13.4 13.4 13.0 13.6 13.5 13.1 13.8
HARRIS 12.4 5.9 17.5 13.7 12.8 13.4 13.0 11.9 12.3 12.2
FAST 1.4 1.1 4.0 4.0 4.0 4.0 4.0 4.1 3.6 3.6
BRISK 27.8 25.3 27.1 25.5 25.6 28.0 25.1 25.3 25.3 29.6
ORB 121.9 3.9 12.5 12.1 11.4 12.8 12.3 13.8 12.7 12.9
AKAZE 62.8 26.2 72.6 47.8 73.4 47.2 64.5 53.0 59.7 65.3
SIFT 91.2 58.0 89.4 86.1 84.4 88.8 86.4 87.7 85.4 86.1

Detector / Keypoints on the Preceding Vehicle [ms]

Detector / Keypoints 0 1 2 3 4 5 6 7 8 9
SHITOMASI 125 118 123 120 120 113 114 123 111 112
HARRIS 50 54 53 55 56 58 57 61 59 57
FAST 419 427 404 423 386 414 418 406 396 401
BRISK 254 274 276 275 293 275 289 268 259 250
ORB 91 102 106 113 109 124 129 127 124 125
AKAZE 162 157 159 154 162 163 173 175 175 175
SIFT 137 131 121 135 134 139 136 147 156 135

This table show the number of keypoints detected on the preceding vehicle for each detector.

Detector, Extractor / Total Matches

Detector, Extractor / Total Matches 0 1 2 3 4 5 6 7 8 9
SHITOMASI, BRISK 0 95 88 80 90 82 79 85 86 82
SHITOMASI, BRIEF 0 115 111 104 101 102 102 100 109 100
SHITOMASI, ORB 0 104 103 100 102 103 98 98 102 97
SHITOMASI, FREAK 0 90 88 87 89 83 78 81 86 84
HARRIS, BRISK 0 43 39 45 44 37 41 44 52 48
HARRIS, BRIEF 0 48 49 47 51 53 49 52 56 55
HARRIS, ORB 0 46 44 48 51 48 50 50 58 54
HARRIS, FREAK 0 43 45 44 40 42 47 43 48 51
FAST, BRISK 0 256 243 241 239 215 251 248 243 247
FAST, BRIEF 0 320 332 299 331 276 327 324 315 307
FAST, ORB 0 306 314 295 318 284 312 323 306 304
FAST, FREAK 0 251 250 228 252 234 269 252 243 246
BRISK, BRISK 0 168 169 157 170 171 186 174 167 182
BRISK, BRIEF 0 174 195 182 177 182 193 208 185 179
BRISK, ORB 0 153 164 155 165 150 179 169 173 171
BRISK, FREAK 0 154 173 154 168 157 181 169 175 165
ORB, BRISK 0 73 74 79 85 79 90 88 86 90
ORB, BRIEF 0 49 43 45 59 53 76 67 83 65
ORB, ORB 0 65 69 71 85 91 99 94 91 89
ORB, FREAK 0 41 36 45 47 44 51 52 48 53
AKAZE, BRISK 0 134 124 129 128 130 132 142 144 141
AKAZE, BRIEF 0 137 133 130 130 134 146 150 147 150
AKAZE, ORB 0 127 128 127 114 131 132 136 138 144
AKAZE, FREAK 0 123 127 129 121 123 132 145 146 135
AKAZE, AKAZE 0 135 138 132 126 128 146 147 149 148
SIFT, BRISK 0 63 64 60 65 59 65 64 67 79
SIFT, BRIEF 0 86 76 72 83 69 75 76 69 87
SIFT, FREAK 0 63 70 63 65 63 59 64 65 79

% of Matches on Preceding Vehicle - Most to Least

Detector, Extractor / Match:Descriptions on Preceding Vehicle (%) 0 1 2 3 4 5 6 7 8 9 AVG
HARRIS, BRIEF 0 88.9 92.5 85.5 91.1 91.4 86.0 85.2 94.9 96.5 90.2
SHITOMASI, BRIEF 0 97.5 90.2 86.7 84.2 90.3 89.5 81.3 98.2 89.3 89.7
HARRIS, ORB 0 85.2 83.0 87.3 91.1 82.8 87.7 82.0 98.3 94.7 88.0
SHITOMASI, ORB 0 88.1 83.7 83.3 85.0 91.2 86.0 79.7 91.9 86.6 86.2
AKAZE, BRIEF 0 87.3 83.6 84.4 80.2 82.2 84.4 85.7 84.0 85.7 84.2
AKAZE, AKAZE 0 86.0 86.8 85.7 77.8 78.5 84.4 84.0 85.1 84.6 83.7
AKAZE, BRISK 0 85.4 78.0 83.8 79.0 79.8 76.3 81.1 82.3 80.6 80.7
HARRIS, FREAK 0 79.6 84.9 80.0 71.4 72.4 82.5 70.5 81.4 89.5 79.1
AKAZE, FREAK 0 78.3 79.9 83.8 74.7 75.5 76.3 82.9 83.4 77.1 79.1
AKAZE, ORB 0 80.9 80.5 82.5 70.4 80.4 76.3 77.7 78.9 82.3 78.9
FAST, BRIEF 0 74.9 82.2 70.7 85.8 66.7 78.2 79.8 79.5 76.6 77.2
HARRIS, BRISK 0 79.6 73.6 81.8 78.6 63.8 71.9 72.1 88.1 84.2 77.1
FAST, ORB 0 71.7 77.7 69.7 82.4 68.6 74.6 79.6 77.3 75.8 75.3
SHITOMASI, BRISK 0 80.5 71.5 66.7 75.0 72.6 69.3 69.1 77.5 73.2 72.8
SHITOMASI, FREAK 0 76.3 71.5 72.5 74.2 73.5 68.4 65.9 77.5 75.0 72.7
ORB, ORB 0 63.7 65.1 62.8 78.0 73.4 76.7 74.0 73.4 71.2 70.9
ORB, BRISK 0 71.6 69.8 69.9 78.0 63.7 69.8 69.3 69.4 72.0 70.4
BRISK, BRIEF 0 63.5 70.7 66.2 60.4 66.2 66.8 77.6 71.4 71.6 68.3
BRISK, BRISK 0 61.3 61.2 57.1 58.0 62.2 64.4 64.9 64.5 72.8 62.9
BRISK, FREAK 0 56.2 62.7 56.0 57.3 57.1 62.6 63.1 67.6 66.0 61.0
FAST, FREAK 0 58.8 61.9 53.9 65.3 56.5 64.4 62.1 61.4 61.3 60.6
BRISK, ORB 0 55.8 59.4 56.4 56.3 54.5 61.9 63.1 66.8 68.4 60.3
FAST, BRISK 0 60.0 60.1 57.0 61.9 51.9 60.0 61.1 61.4 61.6 59.4
SIFT, BRIEF 0 65.6 62.8 53.3 61.9 49.6 55.1 51.7 44.2 64.4 56.5
ORB, BRIEF 0 48.0 40.6 39.8 54.1 42.7 58.9 52.8 66.9 52.0 50.7
SIFT, FREAK 0 48.1 57.9 46.7 48.5 45.3 43.4 43.5 41.7 58.5 48.2
SIFT, BRISK 0 48.1 52.9 44.4 48.5 42.4 47.8 43.5 42.9 58.5 47.7
ORB, FREAK 0 40.2 34.0 39.8 43.1 35.5 39.5 40.9 38.7 42.4 39.4

This table gives an idea about which combo would be best to match most keypoints detected on the preceding vehicle between sequential frames.

The most accurate detector and descriptor combo to use is HARRIS and BRIEF with an average of 90.2% of matches on the preceding vehicle.

Detector, Extractor / Description Extraction Time [ms] - Fastest to Slowest

Detector, Extractor / Description Extraction Time [ms] 0 1 2 3 4 5 6 7 8 9 AVG
SIFT, BRIEF 0.4 0.3 0.3 0.3 0.3 0.3 0.3 0.4 0.4 0.3 0.33
BRISK, BRIEF 0.8 0.4 0.4 0.4 0.5 0.4 0.5 0.4 0.4 0.4 0.46
HARRIS, BRIEF 1.5 1.3 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.68
AKAZE, BRIEF 0.6 0.4 0.5 0.4 0.6 0.9 1 0.7 0.8 0.9 0.68
ORB, BRIEF 0.7 0.2 0.8 0.7 0.8 0.8 0.8 0.8 0.7 0.8 0.71
FAST, BRIEF 1.1 1 0.7 0.7 0.6 0.7 0.7 0.7 0.6 0.6 0.74
FAST, ORB 1 0.8 0.7 0.8 0.7 0.8 0.8 0.7 0.7 0.7 0.77
SHITOMASI, BRIEF 1.6 1 0.7 0.7 0.8 0.7 0.7 0.8 0.7 0.7 0.84
HARRIS, BRISK 1.7 0.8 0.8 0.9 0.9 0.9 0.8 0.8 1 0.7 0.93
SIFT, BRISK 1.1 1 0.9 1 1 1 1 1.1 1.1 1 1.02
AKAZE, BRISK 1.2 1.1 1.1 1.2 1.4 1.2 1.2 1.4 1.3 1.2 1.23
SHITOMASI, BRISK 1.8 1.2 1 1.2 1.3 1.3 0.8 1.2 2 1.5 1.33
ORB, BRISK 1.1 0.9 1.3 1.4 1.1 1.6 1.7 1.7 1.6 1.7 1.41
SHITOMASI, ORB 1.7 0.9 1.5 1.6 1.4 1.5 1.6 1.7 1.6 1.5 1.5
HARRIS, ORB 0.5 1.7 1.5 1.4 1.4 1.6 1.5 1.5 1.6 2.3 1.5
BRISK, BRISK 2.1 1.8 1.9 1.9 2 1.9 2 3 1.8 1.7 2.01
BRISK, ORB 2.4 2.4 2.2 2.1 2 2.2 2.2 2.1 2.2 2.3 2.21
FAST, BRISK 3.2 2.9 2.9 2.8 2.4 2.8 2.9 2.5 2.4 2.5 2.73
AKAZE, ORB 3.8 1.4 4 3.3 2.9 3.3 3 4.2 3.6 3.4 3.29
ORB, ORB 2.5 2 6.2 7.3 7.8 7 6.1 5.9 6.2 6.5 5.75
SIFT, FREAK 15.1 16.3 15.2 15 15 14.9 15.1 15 14.9 14.9 15.14
BRISK, FREAK 16.5 16.5 16 15.2 15.2 15.6 15.3 15.2 15.1 15.1 15.57
FAST, FREAK 17.8 17.2 17.8 15.7 15.9 15.5 15.7 15.6 15.7 15.6 16.25
AKAZE, FREAK 16.6 15.3 19.8 17.9 18.6 18.3 23.2 16 27 15.5 18.82
AKAZE, AKAZE 27.7 24.6 24.7 24.6 28.7 28.7 25.8 29.1 25.5 25 26.44
SHITOMASI, FREAK 38.8 14.6 35.6 36.3 36.1 38.3 37 36.2 34.6 37 34.45
ORB, FREAK 16.1 15.5 45.2 42.2 41.4 42.7 41.9 42.4 40.9 42.6 37.09
HARRIS, FREAK 31.5 14.1 42.3 39.8 42.5 42.1 42.9 41.8 41 42.3 38.03

This table shows that BRIEF is the fastest descriptor to use. The fastest detector to use is SIFT followed by BRISK and HARRIS.

For conclusion, the top 3 detector and descriptor combos to use are:

  1. HARRIS and BRIEF
  2. SHITOMASI and BRIEF
  3. HARRIS and ORB