latest/how_to/track_objects/

[giscus[bot]]

latest/how_to/track_objects/

A set of easy-to-use utilities that will come in handy in any computer vision project.

https://supervision.roboflow.com/latest/how_to/track_objects/

[kvnptl]

Hello, can we some other detector instead of yolo?

[onuralpszr]

Hello, can you give us alternative ?

[SkalskiP]

@kvnptl you can connect pretty much any detector you want. We have connectors to all popular frameworks and libraries like Transformers, MMDetection or Detectron2. So any detection model those libraries have you can run with Supervision.

[lange4531]

Hi! Would it work to use the tracking functionality that comes with tracker = sv.ByteTrack()
box_annotator = sv.BoundingBoxAnnotator() but with FastSAM on video data?

[SkalskiP]

Yup. I don't see why not :)

[afifbmazlin]

Can I use this method directly on jetson nano with CSI camera?

[SkalskiP]

We are using supervision on Jetson 100%. Not sure about the CSI camera.

[AlgosDuCouteau]

hi, does supervision support yolov8 version 8.2? or should i stick to yolov8 version 8.1? thank you very much!

[LinasKo]

We aim to support most yolo models coming from ultratlytics. If you can't run Detections.from_ultralytics on model results - let us know!

[AlgosDuCouteau]

thanks, i will give it a shot when have time, thanks again!

[quankun2002]

Hello, How can I track objects and detect if it enter a specified zone and crop it if it does? thank you

[LinasKo]

Hi @quankun2002 👋

Here's a more recent version of the Object counting notebook. It should start you off on object detection and zone counting.
https://github.com/LinasKo/notebooks/blob/fix/modernize-detect-and-count-objects-in-zone/notebooks/how-to-detect-and-count-objects-in-polygon-zone.ipynb

When you have Detections, you can filter them by tracker_id and use the crop_image function to boxes.
Note that detections.xyxy is an array of xyxy values so you'll need to loop over it.

Hope that helps!

[quankun2002]

Thank you for your help last time, I have another question. I would like to time how long an object enter a polygon zone and do something if it is past a specific time frame example: a car enter a no parking area, if it is in there for more than 5 minutes do something

[quankun2002]

I tried to do it using time.time() but it use real life time instead, Is there a way to do it using frame instead thank you very much

[LinasKo]

If the end-result is in real-life time, I do suggest using real-life time in the calculations.

In that case, time.time is indeed the right solution.

[loupspl]

Hi!
If I were to modify this code to pass my own detection results to the tracker, what would be the format of the detection results ?

[loupspl]

Hi @LinasKo!

Sorry, I wasn't clear enough.

I'm trying to use this project with a custom detector, and its data output apparently do not correspond to any of the expected ouput.

I can chose the output of this detector, with the following items: per frame and per detection: [x1, y1, x2, y2, confidence_score, class_id]

My problem is that it is not formatted, not in the way supervision expects it.

I tried a bit of retroengineering, to build an object usable with the connector from_yolo_v5 but without success.

I also tried bypassing it by building my own detection object to feed the tracker.
From my understanding of looking at the code, it must be an object with xyxy, confidence_score and class_id as attributes, formatted as arrays (n,4), (n), (n) ? Or maybe I should give it a single (n, 6) array ?
Also, am I supposed to give a tracker_id attribute, with an empty (n) array to the detection object, so that the tracker updates it ?

So my question would be: how can I build a detection object similar to what should be given to the tracker and supervision if I'm using a detector not expected from supervision that only gives a list of [x1, y1, x2, y2, confidence_score, class_id] per frame ?
Or what is the connectors that takes something that is the closest to my detection output, so that I can just rebuild an array similar to the expected input of it ?

Thanks in advance if you have the answer to this or for correcting me if I'm missguided!

[LinasKo]

So, the tracker should work on sv.Detections objects and nothing else - that's what you need to create. If a connector we have won't work, then you need to construct it directly. You can use the sv.Detections constructor and refer to the variable definitions for the types the types.

If you have [x1, y1, x2, y2, confidence_score, class_id] then you'll need something similar to this:

• xyxy is np.array([[x1, y1, x2, y2]], dtype=np.float32)
• class_id is np.array([class_id], dtype=int)
• confidence is np.array([confidence_score], dtype=np.float32)

The nested arrays are used when there's more than 1 object detected in the same frame. For example, xyxy might be [[x1, y1, x2, y2], [x1, y1, x2, y2]]. Note you'll then need 2 elements in class_id and confidence too.

When you have the values, call detections = sv.Detections(xyxy=xyxy, class_id=class_id, confidence=confidence).
That should work as an input to the tracker.

[loupspl]

Thank you, I was able to implement both the tracker and the traces annotator doing this!

[LinasKo]

That's really awesome to hear!

[SkalskiP]

Awesome work @LinasKo 🙏🏻

[aberg2323]

I found the traceannotator source code which does have a parameter for trace_length but I am unsure on how to actually increase the trace_length in my code

[LinasKo]

Hi @aberg2323,

Let's discuss this in your previous question.
#784 (comment)

[esssyjr]

i am using a model that detect only one class. For the labels part i used >>>labels = [f"#{tracker_id}" for tracker_id in detections.tracker_id]. but am getting an error:
ValueError: The number of labels provided (0) does not match the number of detections (1). Each detection should have a corresponding label. This discrepancy can occur if the labels and detections are not aligned or if an incorrect number of labels has been provided. Please ensure that the labels array has the same length as the Detections object.

[LinasKo]

Hi @esssyjr 👋

I can't respond today, but it will help me later if you can print the detections object and paste the result here.

[esssyjr]

code:

import supervision as sv
from ultralytics import YOLO
import cv2
import numpy as np

video_path = "Desktop/IMG_3647.MOV"
model = YOLO("KEKE_NAPEP.pt") 

polygons = np.array([
[993, 38],[1449, 42],[1897, 446],[1081, 470]
])

video_info = sv.VideoInfo.from_video_path(video_path)
frame_generator =  sv.get_video_frames_generator(video_path)
bounding_box_annotator = sv.BoundingBoxAnnotator(thickness=4)
label_annotator = sv.LabelAnnotator(text_scale = 1, 
                                     text_thickness = 2)
polygon_zone = sv.PolygonZone(polygons)
byte_track = sv.ByteTrack()
for frame in frame_generator:
    result = model(frame)[0]
    detections = sv.Detections.from_ultralytics(result)
    detections = detections[polygon_zone.trigger(detections)]
    detections = byte_track.update_with_detections(detections=detections)
    

     

    labels = [f"{tracker_id}"
             for tracker_id
             in detections.tracker_id]



    annotated_frame = frame.copy()
    annotated_frame = sv.draw_polygon(annotated_frame, polygon = polygons, color=sv.Color.red())
    annotated_frame = bounding_box_annotator.annotate(
        scene = annotated_frame, detections = detections
    )
    annotated_frame = label_annotator.annotate(
        scene = annotated_frame, detections = detections, labels =labels
    )
    resized_frame = cv2.resize(annotated_frame, (1200,960))
    cv2.imshow("annotated_frame", resized_frame)
    if cv2.waitKey(1) == ord("q"):
        break
cv2.destroyAllWindows()

Error message:

0: 384x640 1 keke_napep, 2215.1ms
Speed: 220.5ms preprocess, 2215.1ms inference, 234.2ms postprocess per image at shape (1, 3, 384, 640)
SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.

0: 384x640 1 keke_napep, 270.6ms
Speed: 1.3ms preprocess, 270.6ms inference, 0.0ms postprocess per image at shape (1, 3, 384, 640)

SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
0: 384x640 2 keke_napeps, 373.3ms
Speed: 18.7ms preprocess, 373.3ms inference, 0.0ms postprocess per image at shape (1, 3, 384, 640)

SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
0: 384x640 1 keke_napep, 223.2ms
Speed: 7.9ms preprocess, 223.2ms inference, 1.0ms postprocess per image at shape (1, 3, 384, 640)

SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
0: 384x640 1 keke_napep, 241.3ms
Speed: 9.5ms preprocess, 241.3ms inference, 1.0ms postprocess per image at shape (1, 3, 384, 640)

SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
0: 384x640 2 keke_napeps, 214.2ms
Speed: 4.0ms preprocess, 214.2ms inference, 1.0ms postprocess per image at shape (1, 3, 384, 640)

SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
0: 384x640 1 keke_napep, 204.1ms
Speed: 16.1ms preprocess, 204.1ms inference, 1.0ms postprocess per image at shape (1, 3, 384, 640)

SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
0: 384x640 1 keke_napep, 197.2ms
Speed: 12.4ms preprocess, 197.2ms inference, 0.0ms postprocess per image at shape (1, 3, 384, 640)
SupervisionWarnings: red is deprecated: `Color.red()` is deprecated and will be removed in `supervision-0.22.0`. Use `Color.RED` instead.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[6], line 28
     24 annotated_frame = sv.draw_polygon(annotated_frame, polygon = polygons, color=sv.Color.red())
     25 annotated_frame = bounding_box_annotator.annotate(
     26     scene = annotated_frame, detections = detections
     27 )
---> 28 annotated_frame = label_annotator.annotate(
     29     scene = annotated_frame, detections = detections, labels =labels
     30 )
     31 resized_frame = cv2.resize(annotated_frame, (1200,960))
     32 cv2.imshow("annotated_frame", resized_frame)

File ~\AppData\Roaming\Python\Python312\site-packages\supervision\utils\conversion.py:21, in convert_for_annotation_method.<locals>.wrapper(self, scene, *args, **kwargs)
     18 @wraps(annotate_func)
     19 def wrapper(self, scene: ImageType, *args, **kwargs):
     20     if isinstance(scene, np.ndarray):
---> 21         return annotate_func(self, scene, *args, **kwargs)
     23     if isinstance(scene, Image.Image):
     24         scene = pillow_to_cv2(scene)

File ~\AppData\Roaming\Python\Python312\site-packages\supervision\annotators\core.py:1045, in LabelAnnotator.annotate(self, scene, detections, labels, custom_color_lookup)
   1041 anchors_coordinates = detections.get_anchors_coordinates(
   1042     anchor=self.text_anchor
   1043 ).astype(int)
   1044 if labels is not None and len(labels) != len(detections):
-> 1045     raise ValueError(
   1046         f"The number of labels provided ({len(labels)}) does not match the "
   1047         f"number of detections ({len(detections)}). Each detection should have "
   1048         f"a corresponding label. This discrepancy can occur if the labels and "
   1049         f"detections are not aligned or if an incorrect number of labels has "
   1050         f"been provided. Please ensure that the labels array has the same "
   1051         f"length as the Detections object."
   1052     )
   1054 for detection_idx, center_coordinates in enumerate(anchors_coordinates):
   1055     color = resolve_color(
   1056         color=self.color,
   1057         detections=detections,
   (...)
   1061         else custom_color_lookup,
   1062     )

ValueError: The number of labels provided (0) does not match the number of detections (1). Each detection should have a corresponding label. This discrepancy can occur if the labels and detections are not aligned or if an incorrect number of labels has been provided. Please ensure that the labels array has the same length as the Detections object.

Further Explaination:
In KEKE_NAPEP.pt, there is only one class which is the KEKE_NAPEP.

[LinasKo]

I can't reproduce it on my end.

1. What version of supervision are you using? Did you install it with pip install supervision?
2. Can you add print(detections) on two lines:

• after detections = sv.Detections.from_ultralytics(result)
• after detections = byte_track.update_with_detections(detections=detections)

This is unusual and shouldn't be happening. You can also test it with pip install supervision==0.21.0 to ensure you're using the most recent version.

[Rich2020]

Please, please, please integrate DeepSort!

[LinasKo]

Soon! We've been thinking about it for a while 🙂

[tonmoy-TS]

Any update on adding DeepSORT or other recent trackers ?

[kumarabhisek91]

While tracking object through bytetracker we are facing issue of id swapping. If 2 person cross each other in a video frame, their tracking id is getting swapped. How to tackle this issue. Whether bytetracker has any in-build method to tackle this issue.

[SkalskiP]

Unfortunately, ByteTrack only uses information about the position of the bounding box and does not take into account the appearance of the object. Therefore, I'm afraid it might be too weak to solve your problem. We are currently working on integrating Supervision with other trackers.

[Jaskirat-singh04]

how to get total count of detected objects?

[rolson24]

You could use something like the line_zone() class to count how many object cross a line. Or if you just want the total count of detected objects, you can create a set that stores the tracking ids, and update it every frame. Then you can use the length of it to know how many objects have been detected so far.

[lob117]

When I use the code, some noise appears on top of the results. It seems like the AI is detecting other objects that do not actually appear in the video

[arieltoledo]

Hi, wonder how to detect only persons ? Try it to filter by class id, but still seeing that detects other objects.

    import numpy as np
    import supervision as sv
    from ultralytics import YOLO
    
    model = YOLO("yolov8n.pt")
    box_annotator = sv.BoundingBoxAnnotator()
    tracker = sv.ByteTrack()

    def callback(frame: np.ndarray, _: int) -> np.ndarray:
        results = model(frame)[0]

        detections = sv.Detections.from_ultralytics(results)
    
        person_class_id = 0
        person_detections = detections[detections.class_id == person_class_id]
    
        print(f"Detected persons: {len(person_detections)}")
    
       if len(person_detections) > 0:
           tracked_persons = tracker.update_with_detections(person_detections)
           annotated_frame = box_annotator.annotate(frame.copy(), detections=tracked_persons)
           return annotated_frame
     else:
         return frame

    sv.process_video(
         source_path="video.mp4",
        target_path="output_video.mp4",
        callback=callback
    )

0: 384x640 5 persons, 6 cars, 1 truck, 422.2ms
Speed: 6.1ms preprocess, 422.2ms inference, 2.9ms postprocess per image at shape (1, 3, 384, 640)
Detected persons: 5

[LinasKo]

I cannot reproduce the issue. Here's an example of only keeping the 'car' class. You can see that the truck was not annotated in the final video.

https://colab.research.google.com/drive/14o07nNagifnblw8rMcgrTCkVKRKKNxD8?usp=sharing

[leechaeyoungg]

Can you apply this to OBB detection? I tried several times but still not working

[LinasKo]

That's odd. It should work the same. Are these lengths the same after filtering?

len(detections) == len(detectiokns.data["xyxyxyxy"])