Awesome papers of deep learning on point clouds

This repo collects papers on point cloud deep learning. Note that the stars I give to each paper contain personal bias for my own project, but actually I do appreciate all the works that have been done in this area. For my own purpose, I can't include all the papers that have been published. A more complete paper list since 2017 is here: https://github.com/Yochengliu/awesome-point-cloud-analysis.

1. Feature extractor

• Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models (ICCV 2017), R. Klokov et al. [pdf] ⭐ ⭐ ⭐ ⭐
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (CVPR 2017), C. R. Qi et al. [pdf] [Github] ⭐ ⭐ ⭐ ⭐ ⭐
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (NeurIPS 2017), C. R. Qi et al. [pdf] [Github] ⭐ ⭐ ⭐ ⭐ ⭐
• PointCNN: Convolution On X-Transformed Points (NeurIPS 2018) Y. Li et al, [pdf] [Github] ⭐ ⭐ ⭐
• A-CNN: Annularly Convolutional Neural Networks on Point Clouds (CVPR 2019), A. Komarichev et al. [pdf]
  ⭐ ⭐ ⭐
• Relation-Shape Convolutional Neural Network for Point Cloud Analysis (CVPR 2019), Y. Liu et al. [pdf]
  ⭐ ⭐ ⭐ ⭐

Other useful links

• ModelNet Benchmark

2. Detection

Only geometry as input

Grid-based methods

• Voting for Voting in Online Point Cloud Object Detection (RSS 2015), D. Z. Wang et al. [pdf] ⭐ ⭐ ⭐
• Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networks (ICRA 2017), M. Engelcke et al. [pdf] ⭐ ⭐ ⭐
• 3D fully convolutional network for vehicle detection in point cloud (IROS 2017) B. Li. [pdf] [Github] ⭐ ⭐ ⭐
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection (CVPR 2018), Y. Zhou et al. [pdf]
  ⭐ ⭐ ⭐ ⭐ ⭐
• PIXOR: Real-time 3D Object Detection From Point Clouds (CVPR 2018), B. Yang et al. [pdf] ⭐ ⭐ ⭐ ⭐
• SECOND: Sparsely Embedded Convolutional Detection (Sensors 2018) Y. Yan et al. [pdf] [Github] ⭐ ⭐ ⭐
• PointPillars: Fast Encoders for Object Detection from Point Clouds (CVPR 2019), A. Lang et al. [pdf] [GIthub]
  ⭐ ⭐ ⭐ ⭐ ⭐
• Part-A^2 Net: 3D Part-Aware and Aggregation Neural Network for Object Detection from Point Cloud (ArXiv 2019) S. Shi et al. [pdf] [Github] ⭐ ⭐ ⭐ ⭐

Point-based methods

• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud (CVPR 2019), S. Shi et al. [pdf] [Github] ⭐ ⭐ ⭐ ⭐ ⭐
• Deep Hough Voting for 3D Object Detection in Point Clouds (ICCV 2019) C. R. Qi et al. [pdf] [Github]
  ⭐ ⭐ ⭐ ⭐ ⭐

Combining point-based and grid-based methods

• STD: Sparse-to-Dense 3D Object Detector for Point Cloud (ICCV 2019), Z. Yang et al. [pdf] ⭐ ⭐ ⭐ ⭐
• Fast Point R-CNN (ICCV 2019), Y. Chen et al. [pdf] ⭐ ⭐ ⭐
• PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection (Arxiv 2019) S. Shi et al. [pdf]
  ⭐ ⭐ ⭐ ⭐ ⭐

2D proposal based

• IPOD: Intensive Point-based Object Detector for Point Cloud (ArXiv 2018) Z. Yang et al. [pdf] ⭐ ⭐ ⭐ ⭐
• RoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement ((ArXiv 2018), K. Shin et al. [pdf]
• Frustum PointNets for 3D Object Detection from RGB-D Data (CVPR 2018), C. R. Qi et al. [pdf] [GIthub]
  ⭐ ⭐ ⭐ ⭐ ⭐
• Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection (CVPR 2019), Z. Wang et al. [pdf] ⭐ ⭐ ⭐

Multi-view/multi-sensor/multi-task

• Multi-View 3D Object Detection Network for Autonomous Driving (CVPR 2017), X. Chen et al. [pdf] [Github]
  ⭐ ⭐ ⭐ ⭐
• PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation (CVPR 2018), D. Xu et al. [pdf] ⭐ ⭐ ⭐ ⭐
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection (ECCV 2018), M. Liang et al. [pdf] ⭐ ⭐ ⭐ ⭐
• Multi-Task Multi-Sensor Fusion for 3D Object Detection (CVPR 2019), M. Liang et al. [pdf] ⭐ ⭐ ⭐ ⭐ ⭐
• MVX-Net: Multimodal VoxelNet for 3D Object Detection (ICRA 2019), V. A. Sindagi et al. [pdf] ⭐ ⭐ ⭐ ⭐

Other useful links

• KITTI Leaderboard
• nuScenes Leaderboard

3. Segmentation

• Recurrent Slice Networks for 3D Segmentation of Point Clouds (CVPR 2018), Q. Huang et al. [pdf] [Github]
  ⭐ ⭐ ⭐ ⭐
• SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation (CVPR 2018), W. Wang et al. [pdf] [Github] ⭐ ⭐ ⭐ ⭐
• Associatively Segmenting Instances and Semantics in Point Clouds (CVPR 2019), X. Long et al. [pdf]
  ⭐ ⭐ ⭐ ⭐ ⭐

...(To be completed)

4. Dataset

Note that some of these datasets don't provide point cloud data, which means you need some toolboxes to convert data from mesh or RGB-D images.

Shape understanding

• ModelNet [pdf] [Project]
• ShapeNet [pdf] [Project]

Indoor scenes

• 2D-3D-S [pdf] [Project]
• ScanNet [pdf] [Project]
• SUN RGB-D [pdf] [Project]

Autonomous driving (Lidar point cloud)

• KITTI [pdf] [Project]
• nuScenes [pdf] [Project]
• Waymo Open dataset [Project]