Generalizing Monocular 3D Human Pose Estimation in the Wild

This repository is the implementation of the work presented in:

Luyang Wang, Yan Chen, Zhenhua Guo, Keyuan Qian, Mude Lin, Hongsheng Li, Jimmy S. Ren, Generalizing Monocular 3D Human Pose Estimation in the Wild.(International Conf. on Computer Vision - Workshop on Geometry Meets Deep Learning 2019) Watch Our Video on YouTube.

Dependencies

Tensorflow >= 1.4.1
Pytorch >= 0.3.1
Numpy = 1.14.3
CV2 = 3.4.1

Dataset

You can download our processed datasets in the list. We train the 3D Label Generator with Human3.6M dataset and Unity dataset. In addition, We train the Baseline Network with MPII/LSP/AIChallenger/Human3.6M datasets. Note that we provided the MPII/LSP/AIChallenger/Human3.6M datasets with high-quality 3D labels, available through Google Drive.

• MPII
• LSP
• AIChallenger
• Human3.6M
• Unity
• LeftRight2D

Guidelines

Download the datasets. All the compressed files suffixes are tar.gz.

tar -zxvf xxx.tar.gz

See more details here.

Pre-trained Model

We also provide a model pre-trained on 3D Label Generator and Baseline Network, available through Baidu Cloud.

• 3D Label Generator

Installation

Clone this repository and download our processed datasets.

git clone https://github.com/llcshappy/Monocular-3D-Human-Pose.git

Useage

3D Label Generator

The code of 3D Label Generator was tested with Anaconda Python3.6 and Tensorflow. After install Anaconda and Tensorflow:

Step 1. Open the 3DLabelGen folder:

cd 3DLabelGen/

Step2. Training Stereoscopic View Synthesis Subnetwork

You need to generate the right-view 2D pose.

python2 gen_right.py

Train the subnetwork

./left2right.sh

Step3. Training 3D Pose Reconstruction Subnetwork

Train the subnetwork

./3DPose.sh

Step4. Geometric Search Scheme

See more details of the geometric search scheme in our paper. Please input the action in script 'search_h36m.py'

# Input the action here
action = 'WalkTogether'

Then run this script.

python2 search_h36m.py

Quick Demo

You can run the following code to see the quick demo of the 3D Label Generator.

./demo.sh

Quick Demo

You can run the following code to see the quick demo of our trained Baseline Network.

./demo.sh

Visualization

Citation

@article{wang2019generalizing,
title={Generalizing Monocular 3D Human Pose Estimation in the Wild},
author={Wang, Luyang and Chen, Yan and Guo, Zhenhua and Qian, Keyuan and Lin, Mude and Li, Hongsheng and Ren, Jimmy S},
journal={arXiv preprint arXiv:1904.05512},
year={2019}
}