Skip to content

2 Object Tracking

Jump to bottom
tasuarez edited this page Jul 15, 2019 · 4 revisions

Object Tracking

Introduction

The inputs of this stage are the ROIs from the detection stage.

The outputs of this stage are a new ROI over the tracked objects along with an ID that identifies it.

The basic idea is to use the detection models in order to identify objects on the input feed. We run the inference process on each frame until we have a detection. We then add that ROI to the tracker system and start tracking it over the following frames. We keep running the inference process every once in a while (a variable X amount of frame) to feedback the tracker with the new information:

  1. If there is a new object, add it to the tracking system.

  2. If the tracker lost the object, add it again.

  3. If tracking the new ROI has been moved away enough from the original object, reset the tracker for that object.

  4. If a detected object is about to exit the scene, remove it.

  5. If there is a lost object being tracked, remove it.

Tracking System

Once we have detections, we build a detection vector, containing all the ROIs from the detection phase. We compare that vector with a tracker vector, that contains all of the information for ROIs from the previous frame. At first we implemented dlib’s correlation tracker object to handle each SingleTracker. This consumes additional resources and is not necessary. Dlibs perform a combination of descriptor and classification over each ROI through each frame. Since we have a detection phase running on each frame we replace dlib by completing a comparison between the detected ROIs and the tracked ROIs. We identify a ROI as 'existing' when 2 ROIs have a % of overlap and their centers are near. This reduces time when tracking multiple objects. In addition, as mentioned on the project’s readme file, we have found that using Intel’s TBB libraries increase the speed of 2x factor.

The Tracker.hpp and Tracker.cpp files implement the tracking system.

Class : SingleTracker

This class is to track a single object. One SingleTracker object is assigned to 'One' object.

Class : TrackingSystem

TrackingSystem is the highest-ranking manager in this system. It uses TrackerManager class for smooth tracking. And SingleTracker object will be included in TrackerManager::tracker_vec. In each SingleTracker, SingleTracker::startSingleTracking and SingleTracker::doSingleTracking functions are taking care of tracking each target. TrackingSystem is using these classes properly and handling all expected exceptions.

Problems found

Add this to main README later.

To take advantage of the Intel hardware, we use dlib’s object tracking that makes use of Intel’s MKL library. After installing it, we can check on cmake’s output it was properly linking, but at runtime, some problems regarding OpenMP popped-up, killing the program. To avoid using OpenMP, we installed Intel’s TBB, and it worked properly.

Also, we have to set the environmental variables for TBB, by running:

source /opt/intel/compilers_and_libraries/linux/tbb/bin/tbbvars.sh intel64

You could also add that to your .bashrc after the line that sets up OpenVINO’s variables.

By using MKL+TBB it works twice as fast than with OpenBLAS (from 7-8 to 14-15 fps).

Clone this wiki locally