Siam R-CNN: Visual Tracking by Re-Detection

Paul Voigtlaender, Jonathon Luiten, Philip H.S. Torr, Bastian Leibe

The corresponding project page can be found here: https://www.vision.rwth-aachen.de/page/siamrcnn

This software is written in Python3 and powered by TensorFlow 1.

We borrow a lot of code from TensorPack's Faster R-CNN example: https://github.com/tensorpack/tensorpack/tree/master/examples/FasterRCNN

Installation

Download necessary libraries

Here we will put all external libraries and this repository into /home/${USERNAME}/vision and use pip to install common libraries

mkdir /home/${USERNAME}/vision
cd /home/${USERNAME}/vision

git clone https://github.com/VisualComputingInstitute/SiamR-CNN.git
git clone https://github.com/pvoigtlaender/got10k-toolkit.git
git clone https://github.com/tensorpack/tensorpack.git

cd tensorpack
git checkout d24a9230d50b1dea1712a4c2765a11876f1e193c
cd ..

pip3 install cython
pip3 install tensorflow-gpu==1.15
pip3 install wget shapely msgpack msgpack_numpy tabulate xmltodict pycocotools opencv-python tqdm zmq annoy

Add libraries to your PYTHONPATH

export PYTHONPATH=${PYTHONPATH}:/home/${USERNAME}/vision/got10k-toolkit/:/home/${USERNAME}/vision/tensorpack/

Make Folder for models and logs and download pre-trained model

cd SiamR-CNN/
mkdir train_log
cd train_log
wget --no-check-certificate -r -nH --cut-dirs=2 --no-parent --reject="index.html*" https://omnomnom.vision.rwth-aachen.de/data/siamrcnn/hard_mining3/
cd ..

Evaluation

For evaluation, first set the path to the dataset on which you want to evaluate in tracking/do_tracking.py, e.g.

OTB_2015_ROOT_DIR = '/data/otb2015/'

Then run tracking/do_tracking.py and specify the dataset you want to evaluate on using the main function for this dataset using e.g. --main main_otb

python3 tracking/do_tracking.py --main main_otb

The result will then be written to tracking_data/results/

Training

Download the pre-trained Mask R-CNN model from http://models.tensorpack.com/FasterRCNN/COCO-MaskRCNN-R101FPN9xGNCasAugScratch.npz

Now change the paths to the training datasets in config.py, e.g.

_C.DATA.IMAGENET_VID_ROOT = "/globalwork/data/ILSVRC_VID/ILSVRC/"

there you can also enable and disable different datasets, e.g.

_C.DATA.IMAGENET_VID = True

To run the main training (without hard example mining):

python3 train.py --load /path/to/COCO-R101FPN-MaskRCNN-ScratchGN.npz

Hints about the code

In the code, we sometimes use the terminology "ThreeStageTracker" or three stages. This refers to the Tracklet Dynamic Programming Algorithm (TDPA).

In order to make the code more readable, we removed some parts before publishing. If there's an important feature which you are missing, please write us an email at voigtlaender@vision.rwth-aachen.de

In the current version of the code, the functions to pre-compute the features for hard example mining are not available, but we can share the pre-computed data on request.

References

If you find this code useful, please cite

Siam R-CNN: Visual Tracking by Re-Detection
Paul Voigtlaender, Jonathon Luiten, Philip H.S. Torr, Bastian Leibe.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.