Learning Descriptor Networks for 3D Shape Synthesis and Analysis

This repository contains a tensorflow implementation for the paper "Learning Descriptor Networks for 3D Shape Synthesis and Analysis ". (http://www.stat.ucla.edu/~jxie/3DDescriptorNet/3DDescriptorNet.html)

Requirements

• Python 2.7 or Python 3.3+
• Tensorflow r1.3+
• Install required Python libraries

  pip install numpy scipy

Getting Started

• Clone this repo:

  git clone https://github.com/jianwen-xie/3DDescriptorNet.git
  cd 3DDescriptorNet

• Download volumetric data and save it to ./data directory. The dataset contains 10 categories of voxelizations of ModelNet10.

• Download pretrained models and save it to the cloned directory.

Exp1: 3D object synthesis

• Train the synthesis model on night stand category:

  python train.py --category night_stand --data_dir ./data/volumetric_data/ModelNet10 --output_dir ./output

• Visualize the generated results using the MATLAB code in visualization/visualize.m, e.g.

  addpath('visualization')
  visualize('./output/night_stand/synthesis', 'sample2990.mat')

• Evaluate synthesized results using the evaluation code in ./evaluation

• You can download our synthesized results and test on it.

Exp2: 3D object recovery

• Train the recovery model on sofa category:

  python rec_exp.py --category sofa \
                    --num_epochs 1000 \
                    --batch_size 50 \
                    --step_size 0.07 \
                    --sample_steps 90 

• Test the recovery model:

  1. Download the incomplete data and save it to ./data directory. For each category in volumetric_data, the incomplete data contains: 1) incomplete_test.mat: 70% randomly corrupted testing data 2) masks.mat: The mask to corrupt the testing data. 3. original_test.mat: original testing data for comparison.
  2. You can download our pretrained model to test recovery.
  3. Run recovery on the corrupted data

  python rec_exp.py --test --category sofa \
                    --ckpt pretrained_model/recovery/sofa/sofa.ckpt \
                    --incomp_data_path ./data/incomplete_data \
                    --batch_size 50 \
                    --step_size 0.07 \
                    --sample_steps 90 

Exp3: 3D object super resolution

• Train the super resolution model on toilet category:

  python sr_exp.py --category toilet \
                    --cube_len 64 \
                    --scale 4 \
                    --num_epochs 500 \
                    --batch_size 50 \
                    --step_size 0.01 \
                    --sample_steps 10 

• Test the super resolution model:

  python rec_exp.py --test --category toilet \
                    --ckpt ./output/toilet/checkpoints/model.ckpt-490 \
                    --cube_len 64 \
                    --scale 4 \
                    --batch_size 50 \
                    --step_size 0.01 \
                    --sample_steps 10 

Exp4: 3D object classification

Method	Classification
Geometry Image	88.4%
PANORAMA-NN	91.1%
ECC	90.0%
3D ShapeNets	83.5%
DeepPana	85.5%
SPH	79.8%
VConv-DAE	80.5%
3D-GAN	91.0%
3D DescriptorNet (ours)	92.4%

• Train Classification using Logistic Regression (pretrained model):

  python train_classification.py --classifier_type logistic --ckpt pretrained_models/classification/model.ckpt
  

• Train Classification using SVM:

  python train_classification.py --classifier_type svm --ckpt pretrained_models/classification/model.ckpt
  

References

@inproceedings{3DDesNet,
    title={Learning Descriptor Networks for 3D Shape Synthesis and Analysis},
    author={Xie, Jianwen and Zheng, Zilong and Gao, Ruiqi and Wang, Wenguan and Zhu Song-Chun and Wu, Ying Nian},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2018}
}

For any questions, please contact Jianwen Xie (jianwen@ucla.edu) and Zilong Zheng (zilongzheng0318@ucla.edu).