[Feature] Support PointNet++ Segmentor #528

Wuziyi616 · 2021-05-08T04:00:33Z

Support 3D Segmentor class
Support 3D decode head class
Add configs to support training on ScanNet dataset

Wuziyi616 · 2021-05-08T04:44:14Z

Currently there are several conflicts and workarounds in the code, e.g.:

MMSeg uses MMCV MODEL registry, while MMDet3D hasn't used. Should change to this style after PR [Enhance] Refactor init_weight #378 is merged
MMSeg hasn't finished refactor_init, so currently 3D Seg models use old init function (with pretrained args). Should change to new style after MMSeg finish refactor_init
Workarounds include build_model warping build_detector and build_segmentor, train_model warping train_detector and train_segmentor.

Tai-Wang · 2021-05-10T06:28:02Z

Support 3D Segmentor class

Support 3D decode head class

Add configs to support training on ScanNet dataset

Actually you can split it into 3 PRs next time, this one is so large :)

Tai-Wang · 2021-05-10T06:39:30Z

Currently there are several conflicts and workarounds in the code, e.g.:

MMSeg uses MMCV MODEL registry, while MMDet3D hasn't used. Should change to this style after PR [Enhance] Refactor init_weight #378 is merged

MMSeg hasn't finished refactor_init, so currently 3D Seg models use old init function (with pretrained args). Should change to new style after MMSeg finish refactor_init

Workarounds include build_model warping build_detector and build_segmentor, train_model warping train_detector and train_segmentor.

build_model and train_model will be used as the workaround for a long time? Or is there any refactor in progress that will resolve this problem?

mmdet3d/models/builder.py

mmdet3d/models/decode_heads/decode_head.py

mmdet3d/models/segmentors/base.py

mmdet3d/models/segmentors/encoder_decoder.py

Tai-Wang

Need to resolve conflicts

Wuziyi616 · 2021-05-10T08:23:03Z

mmdet3d/datasets/custom_3d_seg.py

 from .pipelines import Compose
 from .utils import get_loading_pipeline


 @DATASETS.register_module()
+@SEG_DATASETS.register_module()


Need to register dataset twice because in mmseg's train_segmentor(), it also calls build_dataset. This can be resolved in the future if we unify DATASET registry over all repos.

mmdet3d/models/builder.py

mmdet3d/apis/test.py

mmdet3d/models/segmentors/encoder_decoder.py

mmdet3d/utils/logger.py

tests/test_models/test_segmentors.py

tools/train.py

configs/pointnet2/pointnet2_msg_16x2_scannet-3d-20class.py

mmdet3d/models/segmentors/base.py

mmdet3d/models/segmentors/encoder_decoder.py

ZwwWayne · 2021-05-12T11:08:15Z

mmdet3d/models/segmentors/encoder_decoder.py

+        pts_semantic_mask_cat = torch.stack(pts_semantic_mask)
+        return EncoderDecoder.forward_train(
+            self,
+            img=points_cat,


This is a little bit ugly, can we implement a more complete base class and fully based on that class?

mmdet3d/models/segmentors/encoder_decoder.py

* [refactor][API2.0] Add onnx export and jit trace (open-mmlab#419) * first commit * add async call * add new api onnx export and jit trace * add decorator * fix ci * fix torchscript ci * fix loader * better pipemanager * remove comment, better import * add kwargs * remove comment * better pipeline manager * remove print * [Refactor][API2.0] Api partition calibration (open-mmlab#433) * first commit * add async call * add new api onnx export and jit trace * add decorator * fix ci * fix torchscript ci * fix loader * better pipemanager * remove comment, better import * add partition * move calibration * Better create_calib_table * better deploy * add kwargs * remove comment * better pipeline manager * rename api, remove reduant variable, and misc * [Refactor][API2.0] Api ncnn openvino (open-mmlab#435) * first commit * add async call * add new api onnx export and jit trace * add decorator * fix ci * fix torchscript ci * fix loader * better pipemanager * remove comment, better import * add ncnn api * finish ncnn api * add openvino support * add kwargs * remove comment * better pipeline manager * merge fix * merge util and onnx2ncnn * fix docstring * [Refactor][API2.0] API for TensorRT (open-mmlab#519) * first commit * add async call * add new api onnx export and jit trace * add decorator * fix ci * fix torchscript ci * fix loader * better pipemanager * remove comment, better import * add partition * move calibration * Better create_calib_table * better deploy * add kwargs * remove comment * Add tensorrt API * better pipeline manager * add tensorrt new api * remove print * rename api, remove reduant variable, and misc * add docstring * [Refactor][API2.0] Api ppl other (open-mmlab#528) * first commit * add async call * add new api onnx export and jit trace * add decorator * fix ci * fix torchscript ci * fix loader * better pipemanager * remove comment, better import * add kwargs * Add new APIS for pplnn sdk and misc * remove comment * better pipeline manager * merge fix * update tools/onnx2pplnn.py * rename function

Wuziyi616 added the WIP label May 8, 2021

Wuziyi616 added 12 commits May 8, 2021 21:48

build BaseSegmentor for point sem seg

b7ad8a5

add encoder-decoder segmentor

d06daab

update mmseg dependency

3592955

fix linting errors

862aabe

warp predicted seg_mask in dict

fe9a5d3

add unit test

6c33dc2

use build_model to wrap detector and segmentor

545cc87

fix compatibility with mmseg

e9f36d0

faster sliding inference

9d190d2

merge master

e027c91

configs for training on ScanNet

e14c26c

fix CI errors

f02013f

Tai-Wang reviewed May 10, 2021

View reviewed changes

mmdet3d/models/builder.py Outdated Show resolved Hide resolved

Tai-Wang reviewed May 10, 2021

View reviewed changes

mmdet3d/models/decode_heads/decode_head.py Outdated Show resolved Hide resolved

Tai-Wang reviewed May 10, 2021

View reviewed changes

mmdet3d/models/segmentors/base.py Outdated Show resolved Hide resolved

Tai-Wang reviewed May 10, 2021

View reviewed changes

mmdet3d/models/segmentors/base.py Outdated Show resolved Hide resolved

Tai-Wang reviewed May 10, 2021

View reviewed changes

mmdet3d/models/segmentors/encoder_decoder.py Outdated Show resolved Hide resolved

Tai-Wang reviewed May 10, 2021

View reviewed changes

mmdet3d/models/segmentors/encoder_decoder.py Show resolved Hide resolved

Tai-Wang reviewed May 10, 2021

View reviewed changes

mmdet3d/models/segmentors/encoder_decoder.py Outdated Show resolved Hide resolved

Tai-Wang reviewed May 10, 2021

View reviewed changes

Wuziyi616 force-pushed the pn2_segmentor branch from 6697c87 to f02013f Compare May 10, 2021 07:45

Wuziyi616 removed the WIP label May 10, 2021

Wuziyi616 added 2 commits May 10, 2021 16:10

add comments & fix typos

5f79eb5

hard-code class_weight into configs

bb43330

Wuziyi616 commented May 10, 2021

View reviewed changes

mmdet3d/models/builder.py Show resolved Hide resolved

Wuziyi616 mentioned this pull request May 10, 2021

[Feature] Support 3D semantic segmentation demo #532

Merged

fix logger bugs

e3e52ec

Wuziyi616 requested review from ZwwWayne and Tai-Wang May 11, 2021 09:58

ZwwWayne reviewed May 12, 2021

View reviewed changes

mmdet3d/apis/test.py Show resolved Hide resolved

ZwwWayne reviewed May 12, 2021

View reviewed changes

mmdet3d/apis/test.py Show resolved Hide resolved

ZwwWayne reviewed May 12, 2021

View reviewed changes

mmdet3d/models/segmentors/encoder_decoder.py Outdated Show resolved Hide resolved

ZwwWayne reviewed May 12, 2021

View reviewed changes

mmdet3d/models/segmentors/encoder_decoder.py Outdated Show resolved Hide resolved

ZwwWayne reviewed May 12, 2021

View reviewed changes

mmdet3d/utils/logger.py Outdated Show resolved Hide resolved

ZwwWayne reviewed May 12, 2021

View reviewed changes

tests/test_models/test_segmentors.py Outdated Show resolved Hide resolved

ZwwWayne reviewed May 12, 2021

View reviewed changes

tools/train.py Outdated Show resolved Hide resolved

Wuziyi616 added 4 commits May 12, 2021 13:53

update segmentor unit test

6d3fd1e

logger use mmdet3d

64aa32f

use eps to replace hard-coded 1e-3

c8a8b97

add comments

d40d025

Tai-Wang reviewed May 12, 2021

View reviewed changes

configs/pointnet2/pointnet2_msg_16x2_scannet-3d-20class.py Show resolved Hide resolved

Tai-Wang reviewed May 12, 2021

View reviewed changes

mmdet3d/models/segmentors/base.py Outdated Show resolved Hide resolved

Wuziyi616 added 3 commits May 12, 2021 15:23

replace np operation with torch code

193a566

add comments for class_weight

459e9e5

add comment for BaseSegmentor.simple_test

7e8aef6

Wuziyi616 requested review from Tai-Wang and ZwwWayne May 12, 2021 10:43

ZwwWayne reviewed May 12, 2021

View reviewed changes

mmdet3d/models/segmentors/encoder_decoder.py Outdated Show resolved Hide resolved

ZwwWayne reviewed May 12, 2021

View reviewed changes

mmdet3d/models/segmentors/encoder_decoder.py Outdated Show resolved Hide resolved

rewrite EncoderDecoder3D to avoid inheriting from mmseg

d0bbbf2

ZwwWayne approved these changes May 12, 2021

View reviewed changes

ZwwWayne merged commit 9d852f1 into open-mmlab:master May 12, 2021

Wuziyi616 deleted the pn2_segmentor branch May 13, 2021 04:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support PointNet++ Segmentor #528

[Feature] Support PointNet++ Segmentor #528

Wuziyi616 commented May 8, 2021

Wuziyi616 commented May 8, 2021

Tai-Wang commented May 10, 2021

Tai-Wang commented May 10, 2021

Tai-Wang left a comment

Wuziyi616 May 10, 2021

ZwwWayne May 12, 2021

[Feature] Support PointNet++ Segmentor #528

[Feature] Support PointNet++ Segmentor #528

Conversation

Wuziyi616 commented May 8, 2021

Wuziyi616 commented May 8, 2021

Tai-Wang commented May 10, 2021

Tai-Wang commented May 10, 2021

Tai-Wang left a comment

Choose a reason for hiding this comment

Wuziyi616 May 10, 2021

Choose a reason for hiding this comment

ZwwWayne May 12, 2021

Choose a reason for hiding this comment