Inference or visualization in the wild

[SeanLiu081]

Hi,
thanks for open-sourcing such a great research.I know the PROX recoding data contains depth and mask gt, as well as camera parameters, but other wild videos (such as my selfie video) doesn't have these information,can your method output a visualization？
Or I need to use other pre-trained models to get these information (such as openpose,deeplabv3) , and then run your optimization pipline?

[sanweiliti]

Hi,

Yes you also need depth/human mask/keypoints to run the optimization pipeline, and you can use other open-source tools to get them as you mentioned.

[lucasjinreal]

@sanweiliti hi, may I ask the 3d pose point predicted from 3d pose model such VIBE or PARE can be used for any stage from LEMO?

[sanweiliti]

@jinfagang
Hi, if you want to smooth the 3D joints from VIBE/PARE, you can create an optimization pipeline and insert the motion smoothness prior the same way as we did here, to be specific, in the optimization, you can include the 3D joint loss (to enforce 3D joints to be consistent with VIBE/PARE outputs), motion smoothness loss, and other regularizors (such as shape/pose priors).