[Feature] Add panoptic segmentation loading.

[xizaoqu]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Add panoptic loading in LoadAnnotations3D for semanticKITTI and nuScenes.

Modification

Add _load_panoptic_seg_3d in LoadAnnotations3D. It simply loads panoptic labels from panoptic paths. The labels of segmentation datasets like semanticKITTI and nuScenes need further processing to get semantic labels and instance labels.

BC-breaking (Optional)

Does the modification introduce changes that break the back-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
3. If the modification has potential influence on downstream projects, this PR should be tested with downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[Xiangxu-0103]

It seems that we need to load panoptic_annotation with seg_3d and mask_3d. We should follow the implementation in MMDetection.

[xizaoqu]

It seems that we need to load panoptic_annotation with seg_3d and mask_3d. We should follow the implementation in MMDetection.

In datasets like semanticKITTI and nuScenes, both semantic labels and instance labels are in the same file, so I want to load them together in one function. For example, labels of semanticKITTI are constructed in the form of (instance ids<<16)+semantic labels.
And there are some questions. When only loading semantic labels, we need to filter out the instance part and do some mapping. When loading panoptic labels, we need to split the labels into semantic and instance parts. I plan to do it in the LoadAnnotations3D function with a new arg offset. What do you think of it?

[Xiangxu-0103]

It seems that we need to load panoptic_annotation with seg_3d and mask_3d. We should follow the implementation in MMDetection.

In datasets like semanticKITTI and nuScenes, both semantic labels and instance labels are in the same file, so I want to load them together in one function. For example, labels of semanticKITTI are constructed in the form of (instance ids<<16)+semantic labels. And there are some questions. When only loading semantic labels, we need to filter out the instance part and do some mapping. When loading panoptic labels, we need to split the labels into semantic and instance parts. I plan to do it in the LoadAnnotations3D function with a new arg offset. What do you think of it?

From my perspective, we need to split the original annotation label into semantic_mask and instance_mask as other datasets do for more common usage such as semantic segmentation task. Of course, more suitable method is also welcomed.

[xizaoqu]

It seems that we need to load panoptic_annotation with seg_3d and mask_3d. We should follow the implementation in MMDetection.

In datasets like semanticKITTI and nuScenes, both semantic labels and instance labels are in the same file, so I want to load them together in one function. For example, labels of semanticKITTI are constructed in the form of (instance ids<<16)+semantic labels. And there are some questions. When only loading semantic labels, we need to filter out the instance part and do some mapping. When loading panoptic labels, we need to split the labels into semantic and instance parts. I plan to do it in the LoadAnnotations3D function with a new arg offset. What do you think of it?

From my perspective, we need to split the original annotation label into semantic_mask and instance_mask as other datasets do for more common usage such as semantic segmentation task. Of course, more suitable method is also welcomed.

Do you mean split labels with some preprocess? I think maybe it is not a good idea because it will occupy extra memory and therefore not friendly for user. Maybe I can implement one version of m y method and see if it suits.

[Tai-Wang]

Please merge the latest dev-1.x and resolve the conflicts.

[codecov]

Codecov Report

Base: 46.39% // Head: 47.08% // Increases project coverage by +0.68% 🎉

Coverage data is based on head (89c6ada) compared to base (6810244).
Patch coverage: 88.28% of modified lines in pull request are covered.

❗ Current head 89c6ada differs from pull request most recent head 7ffbb6f. Consider uploading reports for the commit 7ffbb6f to get more accurate results

Additional details and impacted files

@@             Coverage Diff             @@
##           dev-1.x    #2223      +/-   ##
===========================================
+ Coverage    46.39%   47.08%   +0.68%     
===========================================
  Files          259      261       +2     
  Lines        21401    21674     +273     
  Branches      3379     3424      +45     
===========================================
+ Hits          9930    10206     +276     
+ Misses       10827    10815      -12     
- Partials       644      653       +9     

Flag	Coverage Δ
unittests	47.08% <88.28%> (+0.68%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmdet3d/evaluation/__init__.py	100.00% <ø> (ø)
mmdet3d/testing/data_utils.py	70.83% <ø> (ø)
mmdet3d/visualization/local_visualizer.py	33.90% <0.00%> (-0.36%)	⬇️
mmdet3d/datasets/transforms/loading.py	48.13% <59.37%> (+0.79%)	⬆️
mmdet3d/models/segmentors/encoder_decoder.py	17.20% <72.72%> (ø)
mmdet3d/models/segmentors/base.py	57.69% <75.00%> (+5.06%)	⬆️
mmdet3d/evaluation/metrics/panoptic_seg_metric.py	90.00% <90.00%> (ø)
mmdet3d/datasets/dataset_wrappers.py	92.64% <92.59%> (+67.64%)	⬆️
mmdet3d/evaluation/functional/panoptic_seg_eval.py	93.45% <93.45%> (ø)
mmdet3d/datasets/seg3d_dataset.py	68.18% <100.00%> (+0.73%)	⬆️
... and 10 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[sunjiahao1999]

LGTM.