Figure 1. We formulate the problem of Part-Tree-to-Point-Cloud (PT2PC) as a conditional generation task which takes in a symbolic part tree as condition and generates multiple point clouds with shape variations satisfy the structure defined by the part tree.
This paper investigates the novel problem of generating 3D shape point cloud geometry from a symbolic part tree representation. In order to learn such a conditional shape generation procedure in an end-to-end fashion, we propose a conditional GAN "part tree"-to-"point cloud" model (PT2PC) that disentangles the structural and geometric factors.
Our team: Kaichun Mo, He Wang, Xinchen Yan, and Leonidas J. Guibas from Stanford University
Arxiv Version: https://arxiv.org/abs/2003.08624
Project Page: https://cs.stanford.edu/~kaichun/pt2pc/
@article{mo2020pt2pc,
title={{PT2PC}: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions},
author={Mo, Kaichun and Wang, He and Yan, Xinchen and Guibas, Leonidas},
journal={European conference on computer vision (ECCV 2020)},
year={2020}
}
This repository provides data and code as follows.
data/ # contains PartNet data
Chair_hier/ # contains PartNet part-tree JSON files
Chair_geo/ # contains PartNet per-part point cloud geometry
stats/ # contains helper meta-info
part_trees/ # contains part-tree templates
info.txt # each line lists all PartNet shapes sharing the same part-tree template
pt-0/
template.json # the part-tree template JSON file
xxx.txt # for each shape satisfying the template, this file stores the mapping from
# the canonical part ids in template.json to the part ids for shape xxx
# in files data/Chair_hier/xxx.json and data/Chair_geo/xxx.npz
part_semantics/
Chair.txt # stores PartNet semantic part hierarchy
semantics_colors/
Chair.txt # stores the palette used for visualization
log/ # store training logs
metrics/ # store metric-code for the FPD score
fid.py # the main script that computes FPD scores
gt_stats/ # stores the ground-truth statistics for mean and covariance
pointnet_modelnet40/ # code to train PointNet on ModelNet40
train.py # the main training script
trainer.py # the WGAN-gp trainer
model_gen.py # model definition for the generator
model_dis.py # model definition for the discriminator
data.py # the data loader
utils.py # contain utility functions
This code has been tested on Ubuntu 16.04 with Cuda 9.0, GCC 5.4.0, Python 3.6.5 and PyTorch 1.1.0.
Please fill in this form to download the necessary data.
data.zip # put under the root folder and unzip
part_trees.zip # put under stats/ and unzip
gt_stats.zip # put under metrics/ and unzip
pn_ckpt.zip # put under metrics/pointnet_modelnet40/ and unzip
Please run
pip3 install -r requirements.txt
to install the dependencies.
Please also install
cd sampling
python setup.py install
cd ..
cp sampling/build/lib.linux-x86_64-3.6/sampling_cuda.cpython-36m-x86_64-linux-gnu.so .
Download pretrained models and unzip under the root directory.
Then, you can visualize the generation results using Jupyter Notebook quick_test.ipynb.
Simply run
python ./train.py --category Chair
Check the README in hierinsseg. This is our proposed structure reconstruction metric.
Please post issues for questions and more helps on this Github repo page. We encourage using Github issues instead of sending us emails since your questions may benefit others.
MIT License
- [Aug 17, 2020] Release pre-trained models and add a simple Jupyter Notebook for visualizing the results.
- [May 17, 2020] Release the proposed HierInsSeg score for evaluating shape structure reconstruction.
- [March 27, 2020] Preliminary Data and Code released.
- Release evaluation code.
Please request in Github Issue for more code to release.