From 64b7f28f4f6dc89dfcec21cfbfa87b459f930fbc Mon Sep 17 00:00:00 2001 From: nanoshimarobot Date: Sun, 8 Dec 2024 16:27:17 +0900 Subject: [PATCH 01/14] docs: add ground segmentation document --- docs/use_case/ground_segmentation.en.md | 144 ++++++++++++++++++++++++ docs/use_case/ground_segmentation.ja.md | 143 +++++++++++++++++++++++ 2 files changed, 287 insertions(+) create mode 100644 docs/use_case/ground_segmentation.en.md create mode 100644 docs/use_case/ground_segmentation.ja.md diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md new file mode 100644 index 00000000..fc50668e --- /dev/null +++ b/docs/use_case/ground_segmentation.en.md @@ -0,0 +1,144 @@ +# Evaluate Ground Segmentaion + +Evaluate the function of segmenting and removing ground point clouds from input point clouds. + +## Ground Truth data + +The Ground Truth data required for evaluation can be provided using the following two methods, and each can be used by changing the `Evaluation.Conditions.Method` of the scenario. + +**annotated_rosbag** + +This method involves adding a field to the point cloud data in the bag file to represent semantic labels. + +Synchronize and subscribe to topics before and after ground removal, and evaluate accuracy by comparing the number of points with ground labels. + +In this evaluation framework, the semantic labels are assumed to be recorded in an `INT32` field named `entity_id`. + +**annotated_pcd** + +This method involves adding a field to the point cloud data provided as a dataset (~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/*.pnd.bin) to represent semantic labels. + +Compare the point cloud after ground removal with the point cloud in the `pcd.bin` file, and evaluate accuracy by examining the labels associated with the points in the processed point cloud. + +## Evaluation method + +Launching the file executes the following steps: + +1. Execute launch of evaluation node (`ground_segmentation_evaluator_node`), `logging_simulator.launch` file and `ros2 bag play` command +2. Autoware receives sensor data input from previously prepared rosbag and performs ground point cloud removal within the perception module. +3. Evaluation node subscribes to Autoware's output topics, evaluates the accuracy of ground point cloud removal, and dumps the results to a file +4. When the playback of the rosbag is finished, Autoware's launch is automatically terminated, and the evaluation is completed. + +### Points to note during evaluation + +- **annotated_rosbag mode** + The [sensing module of autoware.universe](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390) needs to be modified as follows: + ```diff + if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { + RCLCPP_ERROR( + get_logger(), + "The pointcloud layout is compatible with PointXYZI. You may be using legacy " + "code/data"); + } + + - return; + + // return; <- comment out! + } + ``` + +- **annotated_pcd mode** + Since the evaluation process takes time, the playback rate of the rosbag needs to be reduced. + Example: + `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` + +## Evaluation result + +The results are calculated for each subscription. The format and available states are described below. + +### Normal + +If the Accuracy obtained through the evaluation meets or exceeds the Evaluation.Conditions.accuracy_min specified in the scenario, it is assumed to be normal. + +### Error + +When the normal condition is not met + +## Topic name and data type used by evaluation node + +Subscribed topics: + +| topic name | Data type | +| -------------------------------------------------------------------- | ----------------------------------- | +| /sensing/lidar/concatenated/pointcloud 　　 | sensor_msgs/msg/PointCloud2 | +| /perception/obstacle_segmentation/single_frame/pointcloud | sensor_msgs/msg/PointCloud2 | + +**NOTE: the`/perception/obstacle_segmentation/single_frame/pointcloud`topic can be modified by changing the `evaluation_target_topic` launch argument.** + +Published topics: + +| topic name | Data type | +| ------------------------------------------------------ | ------------------------------ | +| - | - | + +## Arguments passed to logging_simulator.launch + +- localization: false +- planning: false +- control: false +- perception_mode: lidar + +## simulation + +State the information required to run the simulation. + +### Topic to be included in the input rosbag + +| topic name | Data type | +| ---------------------------------- | -------------------------------------------- | +| /sensing/lidar/concatenated/pointcloud | sensor_msgs/msg/PointCloud2 | +| /tf | tf2_msgs/msg/TFMessage | + + +### Topics that must not be included in the input rosbag + +| topic name | Data type | +| -------- | ----------------------- | +| /clock | rosgraph_msgs/msg/Clock | + +The clock is output by the --clock option of ros2 bag play, so if it is recorded in the bag itself, it is output twice, so it is not included in the bag. + +## evaluation + +State the information necessary for the evaluation. + +### Scenario Format + +See [sample](https://github.com/tier4/driving_log_replayer_v2/blob/develop/sample/ground_segmentation/scenario.ja.yaml) + +### Evaluation Result Format + +See [sample](https://github.com/tier4/driving_log_replayer_v2/blob/develop/sample/ground_segmentation/result.json) + +In ground segmentation, the evaluation results for Accuracy, Precision, Recall, Specificity, and F1-score are output for each frame. + +The format of each frame and the metrics format are shown below. +**NOTE: common part of the result file format, which has already been explained, is omitted.** + +```json +{ + "GroundSegmentation": { + "Result": { "Total": "Success or Fail", "Frame": "Success or Fail" }, + "Info": { + "TP": "The number of ground points recognized as ground", + "FP": "The number of obstacle points recognized as ground", + "TN": "The number of obstacle points recognized as obstacle", + "FN": "The number of ground points recognized as obstacle", + "Accuracy": "Accuracy value", + "Precision": "Precision value", + "Recall": "Recall value", + "Specificity": "Specificit value", + "F1-score": "F1-score value`" + } + } +} +``` \ No newline at end of file diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md new file mode 100644 index 00000000..d8af6a92 --- /dev/null +++ b/docs/use_case/ground_segmentation.ja.md @@ -0,0 +1,143 @@ +# 地面点群除去の評価 + +入力点群に対して地面点群をセグメンテーションし、除去する機能について評価する。 + +## Ground Truthデータ + +評価のために必要となるGround Truthデータは以下の2種類の方法で与えることが可能であり、それぞれシナリオの`Evaluation.Conditions.Method`を変更することにより使用できる。 + +**annotated_rosbag** + +bagデータに含まれる点群データに、セマンティックラベルを表すフィールドを持たせる方法。 + +地面点群除去前後の topic を同期 subscribe し、地面ラベルを持つ点数の比較により精度評価を行う。 + +本評価基盤では、セマンティックラベルは`INT32`型の`entity_id`フィールドに記述されていることが前提となっている。 + +**annotated_pcd** + +データセットとして与える点群データ(`~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/*.pnd.bin`)に、セマンティックラベルを表すフィールドを持たせる方法。 + +地面点群除去処理後の点群と、pcd.binファイルに含まれる点群同士を比較し、処理後点群が持つラベルを見ることで精度評価を行う。 + +## 評価方法 + +launch を立ち上げると以下のことが実行され、評価される。 + +1. launch で評価ノード(`ground_segmentation_evaluator_node`)と `logging_simulator.launch`、`ros2 bag play`コマンドを立ち上げる +2. bag から出力されたセンサーデータを autoware が受け取って、perceptionモジュール内で地面点群除去を行う +3. 評価ノードが topic を subscribe して、地面点群の除去精度などについて評価し結果をファイルに記録する +4. bag の再生が終了すると自動で launch が終了して評価が終了する + +### 評価時の注意点 + +- **annotated_rosbagモード** + [autoware.universeのsensingモジュール](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390)を以下のように書き換える必要がある。 + ```diff + if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { + RCLCPP_ERROR( + get_logger(), + "The pointcloud layout is compatible with PointXYZI. You may be using legacy " + "code/data"); + } + + - return; + + // return; <- comment out! + } + ``` + +- **annotated_pcdモード** + 評価処理に時間がかかるため、rosbagの再生レートを下げる必要がある。 + `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` + +## 評価結果 + +topic の subscribe 1 回につき、以下に記述する判定結果が出力される。 + +### 正常 + +評価により得られたAccuracyがシナリオに記述されている`Evaluation.Conditions.accuracy_min`以上の場合、正常とする。 + +### 異常 + +正常の条件を満たさないとき、異常とする。 + +## 評価ノードが使用する Topic 名とデータ型 + +Subscribed topics: + +| topic 名 | データ型 | +| -------------------------------------------------------------------- | ----------------------------------- | +| /sensing/lidar/concatenated/pointcloud 　　 | sensor_msgs/msg/PointCloud2 | +| /perception/obstacle_segmentation/single_frame/pointcloud | sensor_msgs/msg/PointCloud2 | + +**注:`/perception/obstacle_segmentation/single_frame/pointcloud`topicは、launch引数`evaluation_target_topic`で変更可能である。** + +Published topics: + +| topic 名 | データ型 | +| ------------------------------------------------------ | ------------------------------ | +| - | - | + +## logging_simulator.launch に渡す引数 + +- localization: false +- planning: false +- control: false +- perception_mode: lidar + +## simulation + +シミュレーション実行に必要な情報を述べる。 + +### 入力 rosbag に含まれるべき topic + +| topic 名 | データ型 | +| ---------------------------------- | -------------------------------------------- | +| /sensing/lidar/concatenated/pointcloud | sensor_msgs/msg/PointCloud2 | +| /tf | tf2_msgs/msg/TFMessage | + + +### 入力 rosbag に含まれてはいけない topic + +| topic 名 | データ型 | +| -------- | ----------------------- | +| /clock | rosgraph_msgs/msg/Clock | + +clock は、ros2 bag play の--clock オプションによって出力しているので、bag 自体に記録されていると 2 重に出力されてしまうので bag には含めない + +## evaluation + +評価に必要な情報を述べる。 + +### シナリオフォーマット + +[サンプル](https://github.com/tier4/driving_log_replayer_v2/blob/develop/sample/ground_segmentation/scenario.ja.yaml)参照 + +### 評価結果フォーマット + +[サンプル](https://github.com/tier4/driving_log_replayer_v2/blob/develop/sample/ground_segmentation/result.json)参照 + +ground_segmentation では、Accuracy、Precision、Recall、Specificity、F1-scoreを評価した結果を各 frame 毎に出力する。 + +以下に、評価の例を記述する。 +**注:結果ファイルフォーマットで解説済みの共通部分については省略する。** + +```json +{ + "GroundSegmentation": { + "Result": { "Total": "Success or Fail", "Frame": "Success or Fail" }, + "Info": { + "TP": "地面点として認識された地面点の数", + "FP": "地面点として認識された障害物点の数", + "TN": "障害物点として認識された障害物点の数", + "FN": "障害物点として認識された地面点の数", + "Accuracy": "Accuracyの値", + "Precision": "Precisionの値", + "Recall": "Recallの値", + "Specificity": "Specificitの値", + "F1-score": "F1-scoreの値" + } + } +} +``` \ No newline at end of file From 5ea60bdab194f4ec1f1f80d4c17155e1e74d971b Mon Sep 17 00:00:00 2001 From: nanoshimarobot Date: Sun, 8 Dec 2024 16:29:47 +0900 Subject: [PATCH 02/14] modified --- docs/use_case/ground_segmentation.en.md | 4 ++-- docs/use_case/ground_segmentation.ja.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index fc50668e..9660f54c 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -31,7 +31,7 @@ Launching the file executes the following steps: ### Points to note during evaluation -- **annotated_rosbag mode** +- **annotated_rosbag mode** The [sensing module of autoware.universe](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390) needs to be modified as follows: ```diff if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { @@ -46,7 +46,7 @@ Launching the file executes the following steps: } ``` -- **annotated_pcd mode** +- **annotated_pcd mode** Since the evaluation process takes time, the playback rate of the rosbag needs to be reduced. Example: `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md index d8af6a92..5925b58b 100644 --- a/docs/use_case/ground_segmentation.ja.md +++ b/docs/use_case/ground_segmentation.ja.md @@ -31,7 +31,7 @@ launch を立ち上げると以下のことが実行され、評価される。 ### 評価時の注意点 -- **annotated_rosbagモード** +- **annotated_rosbagモード** [autoware.universeのsensingモジュール](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390)を以下のように書き換える必要がある。 ```diff if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { @@ -46,7 +46,7 @@ launch を立ち上げると以下のことが実行され、評価される。 } ``` -- **annotated_pcdモード** +- **annotated_pcdモード** 評価処理に時間がかかるため、rosbagの再生レートを下げる必要がある。 `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` From ef989375e69f9baa96b23290686d9af1eb9d5be4 Mon Sep 17 00:00:00 2001 From: nanoshimarobot Date: Sun, 8 Dec 2024 16:41:50 +0900 Subject: [PATCH 03/14] modified --- docs/use_case/ground_segmentation.en.md | 74 ++++++++++++------------- docs/use_case/ground_segmentation.ja.md | 62 ++++++++++----------- 2 files changed, 68 insertions(+), 68 deletions(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index 9660f54c..257c0ac6 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -6,7 +6,7 @@ Evaluate the function of segmenting and removing ground point clouds from input The Ground Truth data required for evaluation can be provided using the following two methods, and each can be used by changing the `Evaluation.Conditions.Method` of the scenario. -**annotated_rosbag** +### annotated_rosbag This method involves adding a field to the point cloud data in the bag file to represent semantic labels. @@ -14,9 +14,9 @@ Synchronize and subscribe to topics before and after ground removal, and evaluat In this evaluation framework, the semantic labels are assumed to be recorded in an `INT32` field named `entity_id`. -**annotated_pcd** - -This method involves adding a field to the point cloud data provided as a dataset (~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/*.pnd.bin) to represent semantic labels. +### annotated_pcd + +This method involves adding a field to the point cloud data provided as a dataset (~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/\*.pnd.bin) to represent semantic labels. Compare the point cloud after ground removal with the point cloud in the `pcd.bin` file, and evaluate accuracy by examining the labels associated with the points in the processed point cloud. @@ -32,24 +32,25 @@ Launching the file executes the following steps: ### Points to note during evaluation - **annotated_rosbag mode** - The [sensing module of autoware.universe](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390) needs to be modified as follows: - ```diff - if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { - RCLCPP_ERROR( - get_logger(), - "The pointcloud layout is compatible with PointXYZI. You may be using legacy " - "code/data"); - } - - - return; - + // return; <- comment out! - } - ``` + The [sensing module of autoware.universe](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390) needs to be modified as follows: + + ```diff + if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { + RCLCPP_ERROR( + get_logger(), + "The pointcloud layout is compatible with PointXYZI. You may be using legacy " + "code/data"); + } + + - return; + + // return; <- comment out! + } + ``` - **annotated_pcd mode** - Since the evaluation process takes time, the playback rate of the rosbag needs to be reduced. - Example: - `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` + Since the evaluation process takes time, the playback rate of the rosbag needs to be reduced. + Example: + `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` ## Evaluation result @@ -67,18 +68,18 @@ When the normal condition is not met Subscribed topics: -| topic name | Data type | -| -------------------------------------------------------------------- | ----------------------------------- | -| /sensing/lidar/concatenated/pointcloud 　　 | sensor_msgs/msg/PointCloud2 | -| /perception/obstacle_segmentation/single_frame/pointcloud | sensor_msgs/msg/PointCloud2 | +| topic name | Data type | +| --------------------------------------------------------- | --------------------------- | +| /sensing/lidar/concatenated/pointcloud 　　 | sensor_msgs/msg/PointCloud2 | +| /perception/obstacle_segmentation/single_frame/pointcloud | sensor_msgs/msg/PointCloud2 | **NOTE: the`/perception/obstacle_segmentation/single_frame/pointcloud`topic can be modified by changing the `evaluation_target_topic` launch argument.** Published topics: -| topic name | Data type | -| ------------------------------------------------------ | ------------------------------ | -| - | - | +| topic name | Data type | +| ---------- | --------- | +| - | - | ## Arguments passed to logging_simulator.launch @@ -93,17 +94,16 @@ State the information required to run the simulation. ### Topic to be included in the input rosbag -| topic name | Data type | -| ---------------------------------- | -------------------------------------------- | -| /sensing/lidar/concatenated/pointcloud | sensor_msgs/msg/PointCloud2 | -| /tf | tf2_msgs/msg/TFMessage | - +| topic name | Data type | +| -------------------------------------- | --------------------------- | +| /sensing/lidar/concatenated/pointcloud | sensor_msgs/msg/PointCloud2 | +| /tf | tf2_msgs/msg/TFMessage | ### Topics that must not be included in the input rosbag -| topic name | Data type | -| -------- | ----------------------- | -| /clock | rosgraph_msgs/msg/Clock | +| topic name | Data type | +| ---------- | ----------------------- | +| /clock | rosgraph_msgs/msg/Clock | The clock is output by the --clock option of ros2 bag play, so if it is recorded in the bag itself, it is output twice, so it is not included in the bag. @@ -121,7 +121,7 @@ See [sample](https://github.com/tier4/driving_log_replayer_v2/blob/develop/sampl In ground segmentation, the evaluation results for Accuracy, Precision, Recall, Specificity, and F1-score are output for each frame. -The format of each frame and the metrics format are shown below. +The format of each frame and the metrics format are shown below. **NOTE: common part of the result file format, which has already been explained, is omitted.** ```json @@ -141,4 +141,4 @@ The format of each frame and the metrics format are shown below. } } } -``` \ No newline at end of file +``` diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md index 5925b58b..4cae0d13 100644 --- a/docs/use_case/ground_segmentation.ja.md +++ b/docs/use_case/ground_segmentation.ja.md @@ -6,7 +6,7 @@ 評価のために必要となるGround Truthデータは以下の2種類の方法で与えることが可能であり、それぞれシナリオの`Evaluation.Conditions.Method`を変更することにより使用できる。 -**annotated_rosbag** +### annotated_rosbag bagデータに含まれる点群データに、セマンティックラベルを表すフィールドを持たせる方法。 @@ -14,8 +14,8 @@ bagデータに含まれる点群データに、セマンティックラベル 本評価基盤では、セマンティックラベルは`INT32`型の`entity_id`フィールドに記述されていることが前提となっている。 -**annotated_pcd** - +### annotated_pcd + データセットとして与える点群データ(`~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/*.pnd.bin`)に、セマンティックラベルを表すフィールドを持たせる方法。 地面点群除去処理後の点群と、pcd.binファイルに含まれる点群同士を比較し、処理後点群が持つラベルを見ることで精度評価を行う。 @@ -32,23 +32,24 @@ launch を立ち上げると以下のことが実行され、評価される。 ### 評価時の注意点 - **annotated_rosbagモード** - [autoware.universeのsensingモジュール](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390)を以下のように書き換える必要がある。 - ```diff - if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { - RCLCPP_ERROR( - get_logger(), - "The pointcloud layout is compatible with PointXYZI. You may be using legacy " - "code/data"); - } - - - return; - + // return; <- comment out! - } - ``` + [autoware.universeのsensingモジュール](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390)を以下のように書き換える必要がある。 + + ```diff + if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { + RCLCPP_ERROR( + get_logger(), + "The pointcloud layout is compatible with PointXYZI. You may be using legacy " + "code/data"); + } + + - return; + + // return; <- comment out! + } + ``` - **annotated_pcdモード** - 評価処理に時間がかかるため、rosbagの再生レートを下げる必要がある。 - `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` + 評価処理に時間がかかるため、rosbagの再生レートを下げる必要がある。 + `ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=${scenario_file} play_rate:=0.1` ## 評価結果 @@ -66,18 +67,18 @@ topic の subscribe 1 回につき、以下に記述する判定結果が出力 Subscribed topics: -| topic 名 | データ型 | -| -------------------------------------------------------------------- | ----------------------------------- | -| /sensing/lidar/concatenated/pointcloud 　　 | sensor_msgs/msg/PointCloud2 | -| /perception/obstacle_segmentation/single_frame/pointcloud | sensor_msgs/msg/PointCloud2 | +| topic 名 | データ型 | +| --------------------------------------------------------- | --------------------------- | +| /sensing/lidar/concatenated/pointcloud 　　 | sensor_msgs/msg/PointCloud2 | +| /perception/obstacle_segmentation/single_frame/pointcloud | sensor_msgs/msg/PointCloud2 | **注:`/perception/obstacle_segmentation/single_frame/pointcloud`topicは、launch引数`evaluation_target_topic`で変更可能である。** Published topics: -| topic 名 | データ型 | -| ------------------------------------------------------ | ------------------------------ | -| - | - | +| topic 名 | データ型 | +| -------- | -------- | +| - | - | ## logging_simulator.launch に渡す引数 @@ -92,11 +93,10 @@ Published topics: ### 入力 rosbag に含まれるべき topic -| topic 名 | データ型 | -| ---------------------------------- | -------------------------------------------- | -| /sensing/lidar/concatenated/pointcloud | sensor_msgs/msg/PointCloud2 | -| /tf | tf2_msgs/msg/TFMessage | - +| topic 名 | データ型 | +| -------------------------------------- | --------------------------- | +| /sensing/lidar/concatenated/pointcloud | sensor_msgs/msg/PointCloud2 | +| /tf | tf2_msgs/msg/TFMessage | ### 入力 rosbag に含まれてはいけない topic @@ -140,4 +140,4 @@ ground_segmentation では、Accuracy、Precision、Recall、Specificity、F1-sc } } } -``` \ No newline at end of file +``` From fca2348e0f5d03e07180f01b3d94093a5d1c46a8 Mon Sep 17 00:00:00 2001 From: nanoshimarobot Date: Sun, 8 Dec 2024 16:42:46 +0900 Subject: [PATCH 04/14] typo --- docs/use_case/ground_segmentation.en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index 257c0ac6..d8d42dd1 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -1,4 +1,4 @@ -# Evaluate Ground Segmentaion +# Evaluate Ground Segmentation Evaluate the function of segmenting and removing ground point clouds from input point clouds. From 6c67b0084237026b20edb5b0b4ee97dde7fc61f0 Mon Sep 17 00:00:00 2001 From: nanoshimarobot Date: Sun, 8 Dec 2024 20:41:42 +0900 Subject: [PATCH 05/14] modify url --- docs/use_case/ground_segmentation.en.md | 16 ++++++++-------- docs/use_case/ground_segmentation.ja.md | 16 ++++++++-------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index d8d42dd1..4ecc12df 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -32,18 +32,18 @@ Launching the file executes the following steps: ### Points to note during evaluation - **annotated_rosbag mode** - The [sensing module of autoware.universe](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390) needs to be modified as follows: + The [sensing module of autoware.universe](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L386-L394) needs to be modified as follows: ```diff - if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { - RCLCPP_ERROR( - get_logger(), - "The pointcloud layout is compatible with PointXYZI. You may be using legacy " - "code/data"); - } + if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { + RCLCPP_ERROR( + get_logger(), + "The pointcloud layout is compatible with PointXYZI. You may be using legacy " + "code/data"); + } - return; - + // return; <- comment out! + + //return; } ``` diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md index 4cae0d13..9b3641a1 100644 --- a/docs/use_case/ground_segmentation.ja.md +++ b/docs/use_case/ground_segmentation.ja.md @@ -32,18 +32,18 @@ launch を立ち上げると以下のことが実行され、評価される。 ### 評価時の注意点 - **annotated_rosbagモード** - [autoware.universeのsensingモジュール](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L383-L390)を以下のように書き換える必要がある。 + [autoware.universeのsensingモジュール](https://github.com/autowarefoundation/autoware.universe/blob/main/sensing/autoware_pointcloud_preprocessor/src/filter.cpp#L386-L394)を以下のように書き換える必要がある。 ```diff - if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { - RCLCPP_ERROR( - get_logger(), - "The pointcloud layout is compatible with PointXYZI. You may be using legacy " - "code/data"); - } + if (utils::is_data_layout_compatible_with_point_xyzi(*cloud)) { + RCLCPP_ERROR( + get_logger(), + "The pointcloud layout is compatible with PointXYZI. You may be using legacy " + "code/data"); + } - return; - + // return; <- comment out! + + //return; } ``` From c24841902a11c97c9650d2551a949ee50b0209ca Mon Sep 17 00:00:00 2001 From: Hayato Mizushima Date: Wed, 11 Dec 2024 11:04:44 +0900 Subject: [PATCH 06/14] fix: typo Signed-off-by: Hayato Mizushima --- docs/use_case/ground_segmentation.en.md | 2 +- docs/use_case/ground_segmentation.ja.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index 4ecc12df..9ea527c9 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -136,7 +136,7 @@ The format of each frame and the metrics format are shown below. "Accuracy": "Accuracy value", "Precision": "Precision value", "Recall": "Recall value", - "Specificity": "Specificit value", + "Specificity": "Specificity value", "F1-score": "F1-score value`" } } diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md index 9b3641a1..3e30a830 100644 --- a/docs/use_case/ground_segmentation.ja.md +++ b/docs/use_case/ground_segmentation.ja.md @@ -135,7 +135,7 @@ ground_segmentation では、Accuracy、Precision、Recall、Specificity、F1-sc "Accuracy": "Accuracyの値", "Precision": "Precisionの値", "Recall": "Recallの値", - "Specificity": "Specificitの値", + "Specificity": "Specificityの値", "F1-score": "F1-scoreの値" } } From d83d3726f429bcfe40e01ffeb75ee8585f3a0cbb Mon Sep 17 00:00:00 2001 From: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:06:27 +0900 Subject: [PATCH 07/14] Update docs/use_case/ground_segmentation.ja.md --- docs/use_case/ground_segmentation.ja.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md index 3e30a830..6435bc82 100644 --- a/docs/use_case/ground_segmentation.ja.md +++ b/docs/use_case/ground_segmentation.ja.md @@ -85,6 +85,7 @@ Published topics: - localization: false - planning: false - control: false +- sensing: false - perception_mode: lidar ## simulation From 6cb9226541f4ec34835ec09c58bd9dd9ba3c3bfb Mon Sep 17 00:00:00 2001 From: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:06:34 +0900 Subject: [PATCH 08/14] Update docs/use_case/ground_segmentation.ja.md --- docs/use_case/ground_segmentation.ja.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md index 6435bc82..acff1fca 100644 --- a/docs/use_case/ground_segmentation.ja.md +++ b/docs/use_case/ground_segmentation.ja.md @@ -16,7 +16,7 @@ bagデータに含まれる点群データに、セマンティックラベル ### annotated_pcd -データセットとして与える点群データ(`~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/*.pnd.bin`)に、セマンティックラベルを表すフィールドを持たせる方法。 +データセットとして与える点群データ(`~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/*.pcd.bin`)に、セマンティックラベルを表すフィールドを持たせる方法。 地面点群除去処理後の点群と、pcd.binファイルに含まれる点群同士を比較し、処理後点群が持つラベルを見ることで精度評価を行う。 From 0fe3bc19ab75177ede1bc66db3e0bb62827308c2 Mon Sep 17 00:00:00 2001 From: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:06:42 +0900 Subject: [PATCH 09/14] Update docs/use_case/ground_segmentation.en.md --- docs/use_case/ground_segmentation.en.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index 9ea527c9..7417d784 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -86,6 +86,7 @@ Published topics: - localization: false - planning: false - control: false +- sensing: false - perception_mode: lidar ## simulation From b35c39e8fe5e63fb05f0671575f9add3a14909bd Mon Sep 17 00:00:00 2001 From: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:06:48 +0900 Subject: [PATCH 10/14] Update docs/use_case/ground_segmentation.en.md --- docs/use_case/ground_segmentation.en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index 7417d784..fe03ab22 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -16,7 +16,7 @@ In this evaluation framework, the semantic labels are assumed to be recorded in ### annotated_pcd -This method involves adding a field to the point cloud data provided as a dataset (~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/\*.pnd.bin) to represent semantic labels. +This method involves adding a field to the point cloud data provided as a dataset (~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/\*.pcd.bin) to represent semantic labels. Compare the point cloud after ground removal with the point cloud in the `pcd.bin` file, and evaluate accuracy by examining the labels associated with the points in the processed point cloud. From 4d9c8ea1970832b1726259d167aca03984dcd731 Mon Sep 17 00:00:00 2001 From: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:06:54 +0900 Subject: [PATCH 11/14] Update docs/use_case/ground_segmentation.en.md --- docs/use_case/ground_segmentation.en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index fe03ab22..4605348e 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -10,7 +10,7 @@ The Ground Truth data required for evaluation can be provided using the followin This method involves adding a field to the point cloud data in the bag file to represent semantic labels. -Synchronize and subscribe to topics before and after ground removal, and evaluate accuracy by comparing the number of points with ground labels. +Synchronize and subscribe to topics before and after ground removal, and evaluate the accuracy by comparing the number of points with ground and non-ground labels. In this evaluation framework, the semantic labels are assumed to be recorded in an `INT32` field named `entity_id`. From 7b61c8bb287ff6ee7f515ed2d2851190449c82b6 Mon Sep 17 00:00:00 2001 From: badai nguyen <94814556+badai-nguyen@users.noreply.github.com> Date: Wed, 11 Dec 2024 11:07:01 +0900 Subject: [PATCH 12/14] Update docs/use_case/ground_segmentation.en.md --- docs/use_case/ground_segmentation.en.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index 4605348e..59789ed0 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -1,6 +1,6 @@ # Evaluate Ground Segmentation -Evaluate the function of segmenting and removing ground point clouds from input point clouds. +Evaluate the performance of the Obstacle Segmentation sub-component in Autoware, which is responsible for identifying point clouds originating from obstacle that vehicle should avoid. ## Ground Truth data From 5c564bb877b00a5900a4e8660b5565bd865cdd81 Mon Sep 17 00:00:00 2001 From: Hayato Mizushima Date: Wed, 11 Dec 2024 11:48:56 +0900 Subject: [PATCH 13/14] docs: fix dataset path absolute to relative Signed-off-by: Hayato Mizushima --- docs/use_case/ground_segmentation.en.md | 2 +- docs/use_case/ground_segmentation.ja.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/use_case/ground_segmentation.en.md b/docs/use_case/ground_segmentation.en.md index 59789ed0..6c5e6c8b 100644 --- a/docs/use_case/ground_segmentation.en.md +++ b/docs/use_case/ground_segmentation.en.md @@ -16,7 +16,7 @@ In this evaluation framework, the semantic labels are assumed to be recorded in ### annotated_pcd -This method involves adding a field to the point cloud data provided as a dataset (~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/\*.pcd.bin) to represent semantic labels. +This method involves adding a field to the point cloud data provided as a dataset (`dataset/data/LIDAR_CONCAT/\*.pcd.bin`) to represent semantic labels. Compare the point cloud after ground removal with the point cloud in the `pcd.bin` file, and evaluate accuracy by examining the labels associated with the points in the processed point cloud. diff --git a/docs/use_case/ground_segmentation.ja.md b/docs/use_case/ground_segmentation.ja.md index acff1fca..d33c53bb 100644 --- a/docs/use_case/ground_segmentation.ja.md +++ b/docs/use_case/ground_segmentation.ja.md @@ -16,7 +16,7 @@ bagデータに含まれる点群データに、セマンティックラベル ### annotated_pcd -データセットとして与える点群データ(`~/driving_log_replayer_v2/ground_segmentation/dataset/data/LIDAR_CONCAT/*.pcd.bin`)に、セマンティックラベルを表すフィールドを持たせる方法。 +データセットとして与える点群データ(`dataset/data/LIDAR_CONCAT/*.pcd.bin`)に、セマンティックラベルを表すフィールドを持たせる方法。 地面点群除去処理後の点群と、pcd.binファイルに含まれる点群同士を比較し、処理後点群が持つラベルを見ることで精度評価を行う。 From 4b27f7ef7b410398178877fcbf8a39f3a233f38e Mon Sep 17 00:00:00 2001 From: Hayato Mizushima Date: Wed, 11 Dec 2024 11:50:52 +0900 Subject: [PATCH 14/14] feat: disable sensing Signed-off-by: Hayato Mizushima --- .../driving_log_replayer_v2/launch/ground_segmentation.py | 1 + 1 file changed, 1 insertion(+) diff --git a/driving_log_replayer_v2/driving_log_replayer_v2/launch/ground_segmentation.py b/driving_log_replayer_v2/driving_log_replayer_v2/launch/ground_segmentation.py index e004845e..00fde05b 100644 --- a/driving_log_replayer_v2/driving_log_replayer_v2/launch/ground_segmentation.py +++ b/driving_log_replayer_v2/driving_log_replayer_v2/launch/ground_segmentation.py @@ -20,6 +20,7 @@ """ AUTOWARE_DISABLE = { + "sensing": "false", "localization": "false", "planning": "false", "control": "false",