one shot CNN for both detection and feature extraction

[Grabber]

Is there any code showing the loss function to train the current residual network?
What the authors think about having a single network to do detection and feature extraction? Is it possible?

[nwojke]

The training code is not yet published, but that should be a follow-up at some point.

About detection and feature extraction in a single network: We did some initial experiments where we fine-tuned the final layer of a pre-trained py-faster-rcnn on VOC, but it did not work that well for re-identification / tracking. It would be interesting if you get reasonable results, though.

[Grabber]

@nwojke

Are you using a cosine loss function on the network that is extracting features for re-identification? Could you share it, please?

What I'm thinking about is use object detection patch-features as the input for re-identification network, on a single shot. The object detection features already know how to describe pedestrians, for example. I think it is a better idea than just feeding image pedestrian-patches to a new network.

What do you think?

[nwojke]

Sorry for the late reply. We do use a cosine loss function, but haven't found the time to release the training code yet. I will let you know when we get there. Until then, you could experiment with some other well established loss formulations (e.g., [1,2]) and check how well they perform when providing as input the faster-rcnn features. The tracker supports changing from cosine to Euclidean metric, just change "cosine" to "euclidean" in deep_sort_app.py:162.

[1] https://arxiv.org/abs/1703.07737
[2] https://arxiv.org/abs/1511.05939

[nwojke]

Closing this one down due to inactivity.

[tonmoyborah]

@nwojke Is the code for training on objects other than pedestrians out by now?

[ovgeorge]

Is the code for training on objects other than pedestrians out by now?

[abewley]

Please see @nwojke 's other repo cosine_metric_learning for training code and associated reference.