Face Tracking?

[fullfanta]

I tried your program for face tracking.

First, I assumed that one single image is given continuously like a video.
At the first frame, the alignment was successful because it starts with the mean shape.

But the problem happens from the second frame.
After the first frame, current shape is almost close to the ground truth.
The first stage of regressor is trained with the mean shape and augmented data.
Then, the result of regressor contains large displacement from the ground truth.
Therefore, At the second frame, the shape moves largely by first stage of regressor.
As the frame goes, shape diverges smoothly.

Somebody suggests to start with the mean shape at each frame.
But it sometimes fails if face is rotated largely.

Therefore, I think starting with the previous frame's shape is the best way.

Do you have any experience of idea about this problem of face tracking?

[jwyang]

@fullfanta actually, I trained two models, one for face alignment given face region (i.e., rect), and the other one for face tracking given the shape from previous frame. These two models are trained with different initializations. The first one is trained with face regions detected by your face detector, the other one is trained with face regions of bounding box of ground-truth shape. When conducting tracking, I found initialze with mean shape is better than directly using the final shape predicted from previous frame.

Hope this helps you.