Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Chinese Document of 2_new_data_model #729

Merged
merged 6 commits into from
Sep 15, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/2_new_data_model.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ In this note, we give an example for converting the data into KITTI format.

**Note**: We take Waymo as the example here considering its format is totally different from other existing formats. For other datasets using similar methods to organize data, like Lyft compared to nuScenes, it would be easier to directly implement the new data converter (for the second approach above) instead of converting it to another format (for the first approach above).

### KITTI dataset format
## KITTI dataset format
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

Firstly, the raw data for 3D object detection from KITTI are typically organized as follows, where `ImageSets` contains split files indicating which files belong to training/validation/testing set, `calib` contains calibration information files, `image_2` and `velodyne` include image data and point cloud data, and `label_2` includes label files for 3D detection.

Expand Down
103 changes: 103 additions & 0 deletions docs_zh-CN/2_new_data_model.md
Original file line number Diff line number Diff line change
@@ -1 +1,104 @@
# 2: 在自定义数据集上进行训练

本文将主要介绍如何使用自定义数据集来进行模型的训练和测试,本文以 Waymo 数据集作为示例来说明整个流程。
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

基本步骤如下所示:

1. 准备自定义数据集;
2. 准备配置文件;
3. 在自定义数据集上进行模型的训练、测试和推理。

## 准备自定义数据集

在 MMDetection3D 中有三种方式来自定义一个新的数据集:

1. 将新数据集的数据格式重新组织成已支持的数据集格式;
2. 将新数据集的数据格式重新组织成已支持的一种中间格式;
3. 从头开始创建一个新的数据集。

由于前两种方式比第三种方式更加容易,我们更加建议采用前两种方式来自定义数据集。

在本文中,我们采用第一种方式来将 Waymo 数据集重新组织成 KITTI 数据集的数据格式。

**注意**:考虑到 Waymo 数据集的格式与现有的其他数据集的格式的差别较大,因此本文以该数据集为例来讲解如何自定义数据集,从而方便理解数据集自定义的过程。若需要创建的新数据集与现有的数据集的组织格式较为相似,如 Lyft 数据集和 nuScenes 数据集,采用对数据集的中间格式进行转换的方式(第二种方式)而不是采用对数据格式进行转换的方式(第一种方式)更加简单易行。
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

## KITTI 数据集格式
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

应用于 3D 目标检测的 KITTI 原始数据集的组织方式通常如下所示,其中 `ImageSets` 包含数据集划分文件,用以划分训练集/验证集/测试集,`calib` 包含针对 KITTI 数据集的校准信息文件,`image_2` 和 `velodyne` 分别包含图像数据和点云数据,`label_2` 包含与 3D 目标检测相关的标注文件。
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

```
mmdetection3d
├── mmdet3d
├── tools
├── configs
├── data
│ ├── kitti
│ │ ├── ImageSets
│ │ ├── testing
│ │ │ ├── calib
│ │ │ ├── image_2
│ │ │ ├── velodyne
│ │ ├── training
│ │ │ ├── calib
│ │ │ ├── image_2
│ │ │ ├── label_2
│ │ │ ├── velodyne
```

KITTI 官方提供的目标检测开发[工具包]((https://s3.eu-central-1.amazonaws.com/avg-kitti/devkit_object.zip))详细描述了 KITTI 数据集的标注格式,例如,KITTI 标注格式包含了以下的标注信息:
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

```
#Values Name Description
----------------------------------------------------------------------------
1 type Describes the type of object: 'Car', 'Van', 'Truck',
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved
'Pedestrian', 'Person_sitting', 'Cyclist', 'Tram',
'Misc' or 'DontCare'
1 truncated Float from 0 (non-truncated) to 1 (truncated), where
truncated refers to the object leaving image boundaries
1 occluded Integer (0,1,2,3) indicating occlusion state:
0 = fully visible, 1 = partly occluded
2 = largely occluded, 3 = unknown
1 alpha Observation angle of object, ranging [-pi..pi]
4 bbox 2D bounding box of object in the image (0-based index):
contains left, top, right, bottom pixel coordinates
3 dimensions 3D object dimensions: height, width, length (in meters)
3 location 3D object location x,y,z in camera coordinates (in meters)
1 rotation_y Rotation ry around Y-axis in camera coordinates [-pi..pi]
1 score Only for results: Float, indicating confidence in
detection, needed for p/r curves, higher is better.
```

接下来本文将对 Waymo 数据集原始格式进行转换。
首先需要将下载的 Waymo 数据集的数据文件和标注文件转换到 KITTI 数据集的格式,接着定义一个从 KittiDataset 类继承而来的 WaymoDataset 类,来帮助数据的加载、模型的训练和评估。

具体来说,首先使用[数据转换器](https://github.com/open-mmlab/mmdetection3d/blob/master/tools/data_converter/waymo_converter.py)将 Waymo 数据集转换成 KITTI 数据集的格式,并定义 [Waymo 类]((https://github.com/open-mmlab/mmdetection3d/blob/master/mmdet3d/datasets/waymo_dataset.py))对转换的数据进行处理。因为我们将 Waymo 原始数据集进行预处理并重新组织成 KITTI 数据集的格式,因此可以比较容易实现从 KittiDataset 类到 WaymoDataset 类的继承。需要注意的是,由于 Waymo 数据集有相应的官方评估方法,我们需要在定义新数据类的过程中引入官方评估方法,此时用户可以顺利的转换 Waymo 数据的格式,并使用 `WaymoDataset` 数据类进行模型的训练和评估。
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

更多关于 Waymo 数据集预处理的中间结果的细节,请参照对应的[说明文档](https://mmdetection3d.readthedocs.io/en/latest/tutorials/waymo.html)。


## 准备配置文件

第二步是准备配置文件来帮助数据集的读取和使用,另外,配置文件也能够帮助调整超参数从而获得较优的 3D 检测性能。
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

假设我们想要使用 PointPillars 模型在 Waymo 数据集上实现 3 类的 3D 目标检测:vehicle、cyclist、pedestrian,参照 KITTI 数据集[配置文件](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/datasets/kitti-3d-3class.py)、模型[配置文件](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/models/hv_pointpillars_secfpn_kitti.py)和[整体配置文件]((https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class.py)),我们需要准备[数据集配置文件](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/datasets/waymoD5-3d-3class.py)、[模型配置文件](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/_base_/models/hv_pointpillars_secfpn_waymo.py)),并将这两种文件进行结合得到[整体配置文件](https://github.com/open-mmlab/mmdetection3d/blob/master/configs/pointpillars/hv_pointpillars_secfpn_sbn_2x16_2x_waymoD5-3d-3class.py)。

## 训练一个新的模型
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved
为了使用一个新的配置文件来训练模型,可以通过下面的命令来实现:

```shell
python tools/train.py configs/pointpillars/hv_pointpillars_secfpn_sbn_2x16_2x_waymoD5-3d-3class.py
```

更多的的使用细节,请参考 [Case 1](https://mmdetection3d.readthedocs.io/en/latest/1_exist_data_model.html)。
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved

## 测试和推理

为了测试已经训练好的模型的性能,可以通过下面的命令来实现:

```shell
python tools/test.py configs/pointpillars/hv_pointpillars_secfpn_sbn_2x16_2x_waymoD5-3d-3class.py work_dirs/hv_pointpillars_secfpn_sbn_2x16_2x_waymoD5-3d-3class/latest.pth --eval waymo
```

**注意**:为了使用 Waymo 数据集的评估方法,需要参考[说明文档](https://mmdetection3d.readthedocs.io/en/latest/tutorials/waymo.html)并按照官方指导来准备与评估相关联的文件。

更多有关测试和推理的使用细节,请参考 [Case 1]((https://mmdetection3d.readthedocs.io/en/latest/1_exist_data_model.html))。
wHao-Wu marked this conversation as resolved.
Show resolved Hide resolved