Part2Object

By Cheng Shi, Yulin Zhang, Bin Yang, Jiajin Tang, Yuexin Ma and Sibei Yang

The official PyTorch implementation of the "Part2Object: Hierarchical Unsupervised 3D Instance Segmentation".

README structure

• Installation
• Self-Training and Data-efficient
• Main Result and Available Resources

Roadmap

• Installation
• Data download and Preprocessing
• Pseudo Mask Generation
• Upload Pseudo Mask Result
• Self-Training
• Upload Pretrained Models

Installation

We follow Mask3D to install our environment.

Dependencies

The main dependencies of the project are the following:

python: 3.10.9
cuda: 11.3

You can set up a conda environment as follows

# Some users experienced issues on Ubuntu with an AMD CPU
# Install libopenblas-dev (issue #115, thanks WindWing)
# sudo apt-get install libopenblas-dev

export TORCH_CUDA_ARCH_LIST="6.0 6.1 6.2 7.0 7.2 7.5 8.0 8.6"

conda env create -f environment.yml

conda activate part2object

pip3 install torch==1.12.1+cu113 torchvision==0.13.1+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
pip3 install torch-scatter -f https://data.pyg.org/whl/torch-1.12.1+cu113.html
pip3 install 'git+https://github.com/facebookresearch/detectron2.git@710e7795d0eeadf9def0e7ef957eea13532e34cf' --no-deps

mkdir third_party
cd third_party

git clone --recursive "https://github.com/NVIDIA/MinkowskiEngine"
cd MinkowskiEngine
git checkout 02fc608bea4c0549b0a7b00ca1bf15dee4a0b228
python setup.py install --force_cuda --blas=openblas

cd ..
git clone https://github.com/ScanNet/ScanNet.git
cd ScanNet/Segmentator
git checkout 3e5726500896748521a6ceb81271b0f5b2c0e7d2
make

cd ../../pointnet2
python setup.py install

cd ../../
pip3 install pytorch-lightning==1.7.2

Self-Training and Data-efficient

You can download our generated pseudo-labels here or generate by yourself with our code.

Train & Evaluation

To train or test the results of Part2Object, modify the file paths appropriately and run the following scripts.

sh scripts/scannet/scannet_val.sh

Train data efficient model

After getting the base model trained with pseudo-labeling, you can train the data efficient model by modifying the following script appropriately.

sh scripts/scannet/scannet_df.sh

Main Result and Available Resources

Pseudo Label

Methods	AP25	AP50	mAP
Part2Object	55.1	26.8	12.6	result

Model

Methods	AP50 / (0% data)		AP50 / 1% data		AP50 / 5% data		AP50 / 10% data		AP50 / 20% data
Part2Object	32.6	weight	44.1	weight	64.2	weight	68.0	weight	72.1	weight

Acknowledgement

We thank Mask3D for their valuable code bases.

If you find Part2Object useful in your research, please consider citing:

@article{shi2024part2object,
  title={Part2Object: Hierarchical Unsupervised 3D Instance Segmentation},
  author={Shi, Cheng and Zhang, Yulin and Yang, Bin and Tang, Jiajin and Ma, Yuexin and Yang, Sibei},
  journal={arXiv preprint arXiv:2407.10084},
  year={2024}
}