OpenPCSeg: An Open Source Point Cloud Segmentation Codebase

OpenPCSeg is an open-source point cloud segmentation toolbox based on PyTorch, heading towards the unification and thriving of 3D scene understanding and its related areas. With the help of OpenPCSeg, we benchmark methods in a way that pursues fairness, efficiency, and effectiveness, on prevailing large-scale point cloud datasets. At this moment, OpenPCSeg focuses on outdoor point cloud segmentation for autonomous driving.

This project is developed and maintained by Autonomous Driving Group [at] Shanghai AI Laboratory (ADLab).

Highlight

• 🌈 Comprehensive: We support the full spectrum of point cloud scene understanding tasks on standard large-scale benchmarks, with backbones across various 2D & 3D modalities.
• ⚖️ Fair & Reproducible: We unify the training, validation, and testing configurations, resulting in directly comparable 3D segmentation performance.
• 🚀 Fast & Accurate: Our implementations yield much better segmentation accuracy as well as training and inference speed than the originally reported results.

Supported Task

#	Task	Emphasis	Dataset	Method
🚗	Fully-Supervised LiDAR Segmentation	Accuracy	2	4

Changelog

• [2023.4] - Support sequence-wise OpenPCSeg inference and visualization on Waymo Open Dataset.
• [2023.3] - Initial release of OpenPCSeg v1.0.

Upcoming Features

• Support NuScenes Dataset
• Add implementations of range-view algorithms
• Add toolbox for competition submission
• Add visualiztion of 3D segmentation results

Outline

• Supported Methods
• Supported Benchmark
• Model Zoo
• Installation
• Data Preparation
• Getting Started
• Visualization
• Citation
• Authors
• License
• Contact

Supported Methods

• MinkowskiNet
• Cylinder3D
• SPVCNN
• RPVNet
• More to come...

Supported Benchmark

LiDAR Semantic Segmentation (5)

• SemanticKITTI
• ScribbleKITTI
• Waymo Open
• A2D2 (coming soon)
• nuScenes (coming soon)

LiDAR Panoptic Segmentation (3)

• Panoptic nuScenes (coming soon)
• Panoptic SemanticKITTI (coming soon)
• 4D Panoptic SemanticKITTI (coming soon)

Model Zoo

SemanticKITTI Dataset Representative Models

Selected supported methods are shown in the below table. The results are mean IoU performance of moderate difficulty on the val set of SemanticKITTI dataset.

• All LiDAR-based models are trained with 2 A100 GPUs and are available for download.
• The training and validation are executed with CUDA 11.3 and PyTorch 1.10.
• All models are trained with merely train split of SemanticKITTI Dataset.
• Prediction from our models are directly used for evaluation, without employing any Test Time Augmentation or ensembling.

	training time	mIoU	download
MinkowskiNet	~12.0 hours	70.04	model-737M
Cylinder3D	~8.7 hours	66.07	model-56M
SPVCNN	~13.1 hours	68.58	model-166M
RPVNet	~14.5 hours	68.86	model-980M

Waymo Open Dataset Representative Models

We are not supposed to provide pre-trained weights due to Waymo Dataset License Agreement, but you can easily achieve similar performance by training according to the default configs.

	training time	mIoU
MinkowskiNet	~25.0 hours	69.20
Cylinder3D	~19.2 hours	66.42
SPVCNN	~28.0 hours	69.37

Installation

Please refer to INSTALL.md for the installation details.

Data Preparation

└── data_root 
    └── nuscenes
    └── semantickitti
    └── scribblekitti
    └── waymo_open

Please refer to DATA_PREPARE.md for the details to prepare the nuScenes, SemanticKITTI, ScribbleKITTI, and Waymo Open datasets.

Getting Started

Please refer to GET_STARTED.md to learn more usage about this codebase.

Visualization

We recently supported a OpenPCSeg visualization tool for Waymo Open Dataset, please check INFER_WAYMO.md.

Here are some inference & visualization results:

Citation

If you find this work helpful, please kindly consider citing:

@misc{openpcseg2023,
    title={OpenPCSeg: An Open Source Point Cloud Segmentation Codebase},
    author={Liu Youquan, Bai Yeqi, Kong Lingdong, Chen Runnan, Hou Yuenan, Shi Botian, Li Yikang},
    howpublished = {\url{https://github.com/PJLab-ADG/PCSeg}},
    year={2023}
}

Authors

Liu Youquan, Bai Yeqi†, Kong Lingdong, Chen Runnan, Hou Yuenan, Shi Botian, Li Yikang

†Project Lead

License

OpenPCSeg is released under the Apache 2.0 license, while some specific operations in this codebase might be with other licenses. Please refer to LICENSE for a more careful check, if you are using our code for commercial matters.

Contact

If you have questions about this repo, please contact Bai Yeqi (baiyeqi@pjlab.org.cn) or Shi Botian (shibotian@pjlab.org.cn).

Acknowledgement

The overall structure of this repo is derived from OpenPCDet.

Part of our implementation uses code from repositories below:

• [1] https://github.com/xinge008/Cylinder3D
• [2] https://github.com/mit-han-lab/spvnas
• [3] https://github.com/TiagoCortinhal/SalsaNext
• [4] https://github.com/open-mmlab/OpenPCDet
• [5] https://github.com/mit-han-lab/torchsparse

Thank the authors for their great work!