Skip to content

Commit

Permalink
added torchvision det, seg n pose examples to repo
Browse files Browse the repository at this point in the history
  • Loading branch information
Mikel Broström committed Sep 30, 2024
1 parent 826d4a0 commit 68e2357
Showing 1 changed file with 6 additions and 212 deletions.
218 changes: 6 additions & 212 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,55 +40,6 @@ This repo contains a collections of pluggable state-of-the-art multi-object trac

</details>

<details>
<summary>3rd party resources</summary>

* [ROS wrapper for boxmot (link to external repository)](https://github.com/KalanaRatnayake/boxmot_ros)&nbsp;
* [Yolov10 Integration with BoxMOT (link to external Notebook)](https://colab.research.google.com/drive/1-QV2TNfqaMsh14w5VxieEyanugVBG14V?usp=drive_link)&nbsp;
* [Yolov8 training (link to external repository)](https://docs.ultralytics.com/modes/train/)&nbsp;
* [ReID model training (link to external repository)](https://kaiyangzhou.github.io/deep-person-reid/user_guide.html)&nbsp;
* [ReID model inference acceleration with Nebullvm (link to external Notebook)](https://colab.research.google.com/drive/1APUZ1ijCiQFBR9xD0gUvFUOC8yOJIvHm?usp=sharing)&nbsp;

</details>

<details>
<summary>Experiments</summary>
In inverse chronological order:

* [Impact of Selective Mechanism for Feature Extraction](https://github.com/mikel-brostrom/boxmot/wiki/Impact-of-Selective-Mechanism-for-Feature-Extraction)

* [Evaluation of the params evolved for first half of MOT17 on the complete MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Evaluation-of-the-params-evolved-for-first-half-of-MOT17-on-the-complete-MOT17)

* [Segmentation model vs object detetion model on MOT metrics](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Segmentation-model-vs-object-detetion-model-on-MOT-metrics)

* [Effect of masking objects before feature extraction](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Masked-detection-crops-vs-regular-detection-crops-for-ReID-feature-extraction)

* [conf-thres vs HOTA, MOTA and IDF1](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/conf-thres-vs-MOT-metrics)

* [Effect of KF updates ahead for tracks with no associations on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-KF-updates-ahead-for-tracks-with-no-associations,-on-MOT17)

* [Effect of full images vs 1280 input to StrongSORT on MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Effect-of-passing-full-image-input-vs-1280-re-scaled-to-StrongSORT-on-MOT17)

* [Effect of different OSNet architectures on MOT16](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/OSNet-architecture-performances-on-MOT16)

* [Yolov5 StrongSORT vs BoTSORT vs OCSORT](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/StrongSORT-vs-BoTSORT-vs-OCSORT)
* Yolov5 [BoTSORT](https://arxiv.org/abs/2206.14651) branch: https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/tree/botsort

* [Yolov5 StrongSORT OSNet vs other trackers MOT17](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-17-evaluation-(private-detector))&nbsp;

* [StrongSORT MOT16 ablation study](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/Yolov5DeepSORTwithOSNet-vs-Yolov5StrongSORTwithOSNet-ablation-study-on-MOT16)&nbsp;

* [Yolov5 StrongSORT OSNet vs other trackers MOT16 (deprecated)](https://github.com/mikel-brostrom/Yolov5_StrongSORT_OSNet/wiki/MOT-16-evaluation)&nbsp;

</details>

#### News

* Trackers MOT metrics, on 2nd half of MOT17 training set, updated in README from CI
* Metrics for all trackers generated in CI on CPU by using pre-computed detections and embeddings
* Added YOLOv9 and YOLOv10 support
* Hyperparameter search for all trackers using RayTune (July 2024)
* Added Improved Association Pipeline Tracker by [rolson24](https://github.com/rolson24) in [#1527](https://github.com/mikel-brostrom/boxmot/pull/1527) (July 2024)


## Why BOXMOT?
Expand Down Expand Up @@ -254,172 +205,15 @@ The set of hyperparameters leading to the best HOTA result are written to the tr

## Custom tracking examples

<details>
<summary>Detection</summary>

```python
import cv2
import numpy as np
from pathlib import Path

from boxmot import DeepOCSORT


tracker = DeepOCSORT(
reid_weights=Path('osnet_x0_25_msmt17.pt'), # which ReID model to use
device='cpu',
half=False,
)

vid = cv2.VideoCapture(0)
## Examples

while True:
ret, im = vid.read()
| Example Description | Notebook |
|---------------------|----------|
| Bounding box tracking with BoxMOT using torchvision | [![Notebook](https://img.shields.io/badge/Notebook-torchision_det_boxmot.ipynb-blue)](examples/torchision_det_boxmot.ipynb) |
| Pose tracking with BoxMOT using torchvision | [![Notebook](https://img.shields.io/badge/Notebook-torchvision_pose_boxmot.ipynb-blue)](examples/torchvision_pose_boxmot.ipynb) |
| Segmentation tracking with BoxMOT using torchvision | [![Notebook](https://img.shields.io/badge/Notebook-torchvision_seg_boxmot.ipynb-blue)](examples/torchvision_seg_boxmot.ipynb) |

# substitute by your object detector, output has to be N X (x, y, x, y, conf, cls)
dets = np.array([[144, 212, 578, 480, 0.82, 0],
[425, 281, 576, 472, 0.56, 65]])

# Check if there are any detections
if dets.size > 0:
tracker.update(dets, im) # --> M X (x, y, x, y, id, conf, cls, ind)
# If no detections, make prediction ahead
else:
dets = np.empty((0, 6)) # empty N X (x, y, x, y, conf, cls)
tracker.update(dets, im) # --> M X (x, y, x, y, id, conf, cls, ind)
tracker.plot_results(im, show_trajectories=True)

# break on pressing q or space
cv2.imshow('BoxMOT detection', im)
key = cv2.waitKey(1) & 0xFF
if key == ord(' ') or key == ord('q'):
break

vid.release()
cv2.destroyAllWindows()
```

</details>


<details>
<summary>Pose & segmentation</summary>

```python
import cv2
import numpy as np
from pathlib import Path

from boxmot import DeepOCSORT


tracker = DeepOCSORT(
reid_weights=Path('osnet_x0_25_msmt17.pt'), # which ReID model to use
device='cpu',
half=False,
)

vid = cv2.VideoCapture(0)

while True:
ret, im = vid.read()

keypoints = np.random.rand(2, 17, 3)
mask = np.random.rand(2, 480, 640)
# substitute by your object detector, input to tracker has to be N X (x, y, x, y, conf, cls)
dets = np.array([[144, 212, 578, 480, 0.82, 0],
[425, 281, 576, 472, 0.56, 65]])

tracks = tracker.update(dets, im) # --> M x (x, y, x, y, id, conf, cls, ind)

# xyxys = tracks[:, 0:4].astype('int') # float64 to int
# ids = tracks[:, 4].astype('int') # float64 to int
# confs = tracks[:, 5]
# clss = tracks[:, 6].astype('int') # float64 to int
inds = tracks[:, 7].astype('int') # float64 to int

# in case you have segmentations or poses alongside with your detections you can use
# the ind variable in order to identify which track is associated to each seg or pose by:
# masks = masks[inds]
# keypoints = keypoints[inds]
# such that you then can: zip(tracks, masks) or zip(tracks, keypoints)

# break on pressing q or space
cv2.imshow('BoxMOT segmentation | pose', im)
key = cv2.waitKey(1) & 0xFF
if key == ord(' ') or key == ord('q'):
break

vid.release()
cv2.destroyAllWindows()
```

</details>

<details>
<summary>Tiled inference</summary>

```py
from sahi import AutoDetectionModel
from sahi.predict import get_sliced_prediction
import cv2
import numpy as np
from pathlib import Path
from boxmot import DeepOCSORT


tracker = DeepOCSORT(
reid_weights=Path('osnet_x0_25_msmt17.pt'), # which ReID model to use
device='cpu',
half=False,
)

detection_model = AutoDetectionModel.from_pretrained(
model_type='yolov8',
model_path='yolov8n.pt',
confidence_threshold=0.5,
device="cpu", # or 'cuda:0'
)

vid = cv2.VideoCapture(0)
color = (0, 0, 255) # BGR
thickness = 2
fontscale = 0.5

while True:
ret, im = vid.read()

# get sliced predictions
result = get_sliced_prediction(
im,
detection_model,
slice_height=256,
slice_width=256,
overlap_height_ratio=0.2,
overlap_width_ratio=0.2
)
num_predictions = len(result.object_prediction_list)
dets = np.zeros([num_predictions, 6], dtype=np.float32)
for ind, object_prediction in enumerate(result.object_prediction_list):
dets[ind, :4] = np.array(object_prediction.bbox.to_xyxy(), dtype=np.float32)
dets[ind, 4] = object_prediction.score.value
dets[ind, 5] = object_prediction.category.id

tracks = tracker.update(dets, im) # --> (x, y, x, y, id, conf, cls, ind)

tracker.plot_results(im, show_trajectories=True)

# break on pressing q or space
cv2.imshow('BoxMOT tiled inference', im)
key = cv2.waitKey(1) & 0xFF
if key == ord(' ') or key == ord('q'):
break

vid.release()
cv2.destroyAllWindows()
```

</details>

## Contributors

Expand Down

0 comments on commit 68e2357

Please sign in to comment.