【Hackathon + No.161】论文复现：CLRNet: Cross Layer Refinement Network for L…

…ane Detection (#8278) * Feat add CLRNet * Update CLRNet README.md * Update requirements: add imgaug>=0.4.0 for CLRNet * Update CLRNet README.md * Update README.md * Update Rename clrnet_utils.py * Update CLRNet demo & delete demo result * Update README.md add weight for culane * Update README.cn.md add training logs * Feat add dataset download * Fix bugs when lanes is empty * Update README * Update README for dataset info * Fix export model * Update configs & README * style: update codestyle * Style update op codestyple * Fix eval process * Fix eval process * Update README&configs * Fix deploy infer * Fix mkdir in lane visualize * Docs Update README * Docs Rename configs * Docs update weights --------- Co-authored-by: LokeZhou <aishenghuoaiqq@163.com>
PaddlePaddle · Jun 1, 2023 · 129ddbb · 129ddbb
1 parent a694be1
commit 129ddbb
Show file tree

Hide file tree

Showing 38 changed files with 4,334 additions and 12 deletions.
diff --git a/configs/clrnet/README.cn.md b/configs/clrnet/README.cn.md
@@ -0,0 +1,68 @@
+简体中文 | [English](README.md)
+
+# CLRNet (CLRNet: Cross Layer Refinement Network for Lane Detection)
+
+## 目录
+- [简介](#简介)
+- [模型库](#模型库)
+- [引用](#引用)
+
+## 介绍
+
+[CLRNet](https://arxiv.org/abs/2203.10350)是一个车道线检测模型。CLRNet模型设计了车道线检测的直线先验轨迹，车道线iou以及nms方法，融合提取车道线轨迹的上下文高层特征与底层特征，利用FPN多尺度进行refine，在车道线检测相关数据集取得了SOTA的性能。
+
+## 模型库
+
+### CLRNet在CUlane上结果
+
+| 骨架网络       | mF1 | F1@50   |    F1@75    | 下载链接 | 配置文件 |训练日志|
+| :--------------| :------- |  :----: | :------: | :----: |:-----: |:-----: |
+| ResNet-18         | 54.98 |  79.46  |    62.10   | [下载链接](https://paddledet.bj.bcebos.com/models/clrnet_resnet18_culane.pdparams) | [配置文件](./clrnet_resnet18_culane.yml) |[训练日志](https://bj.bcebos.com/v1/paddledet/logs/train_clrnet_r18_15_culane.log)|
+
+### 数据集下载
+下载[CULane数据集](https://xingangpan.github.io/projects/CULane.html)并解压到`dataset/culane`目录。
+
+您的数据集目录结构如下：
+```shell
+culane/driver_xx_xxframe    # data folders x6
+culane/laneseg_label_w16    # lane segmentation labels
+culane/list                 # data lists
+```
+如果您使用百度云链接下载，注意确保`driver_23_30frame_part1.tar.gz`和`driver_23_30frame_part2.tar.gz`解压后的文件都在`driver_23_30frame`目录下。
+
+现已将用于测试的小数据集上传到PaddleDetection，可通过运行训练脚本，自动下载并解压数据，如需复现结果请下载链接中的全量数据集训练。
+
+### 训练
+- GPU单卡训练
+```shell
+python tools/train.py -c configs/clrnet/clr_resnet18_culane.yml
+```
+- GPU多卡训练
+```shell
+export CUDA_VISIBLE_DEVICES=0,1,2,3
+python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/clrnet/clr_resnet18_culane.yml
+```
+
+### 评估
+```shell
+python tools/eval.py -c configs/clrnet/clr_resnet18_culane.yml -o weights=output/clr_resnet18_culane/model_final.pdparams
+```
+
+### 预测
+```shell
+python tools/infer_culane.py -c configs/clrnet/clr_resnet18_culane.yml -o weights=output/clr_resnet18_culane/model_final.pdparams --infer_img=demo/lane00000.jpg
+```
+
+注意：预测功能暂不支持模型静态图推理部署。
+
+## 引用
+```
+@InProceedings{Zheng_2022_CVPR,
+    author    = {Zheng, Tu and Huang, Yifei and Liu, Yang and Tang, Wenjian and Yang, Zheng and Cai, Deng and He, Xiaofei},
+    title     = {CLRNet: Cross Layer Refinement Network for Lane Detection},
+    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+    month     = {June},
+    year      = {2022},
+    pages     = {898-907}
+}
+```
diff --git a/configs/clrnet/README.md b/configs/clrnet/README.md
@@ -0,0 +1,68 @@
+English | [简体中文](README_cn.md)
+
+# CLRNet (CLRNet: Cross Layer Refinement Network for Lane Detection)
+
+## Table of Contents
+- [Introduction](#Introduction)
+- [Model Zoo](#Model_Zoo)
+- [Citations](#Citations)
+
+## Introduction
+
+[CLRNet](https://arxiv.org/abs/2203.10350) is a lane detection model. The CLRNet model is designed with line prior for lane detection, line iou loss as well as nms method, fused to extract contextual high-level features of lane line with low-level features, and refined by FPN multi-scale. Finally, the model achieved SOTA performance in lane detection datasets.
+
+## Model Zoo
+
+### CLRNet Results on CULane dataset
+
+| backbone       | mF1 | F1@50   |    F1@75    | download | config |
+| :--------------| :------- |  :----: | :------: | :----: |:-----: |
+| ResNet-18         | 54.98 |  79.46  |    62.10   | [model](https://paddledet.bj.bcebos.com/models/clrnet_resnet18_culane.pdparams) | [config](./clrnet_resnet18_culane.yml) |
+
+### Download
+Download [CULane](https://xingangpan.github.io/projects/CULane.html). Then extract them to `dataset/culane`.
+
+For CULane, you should have structure like this:
+```shell
+culane/driver_xx_xxframe    # data folders x6
+culane/laneseg_label_w16    # lane segmentation labels
+culane/list                 # data lists
+```
+If you use Baidu Cloud, make sure that images in `driver_23_30frame_part1.tar.gz` and `driver_23_30frame_part2.tar.gz` are located in one folder `driver_23_30frame` instead of two seperate folders after you decompress them.
+
+Now we have uploaded a small subset of CULane dataset to PaddleDetection for code checking. You can simply run the training script below to download it automatically. If you want to implement the results, you need to download the full dataset at th link for training.
+
+### Training
+- single GPU
+```shell
+python tools/train.py -c configs/clrnet/clr_resnet18_culane.yml
+```
+- multi GPU
+```shell
+export CUDA_VISIBLE_DEVICES=0,1,2,3
+python -m paddle.distributed.launch --gpus 0,1,2,3 tools/train.py -c configs/clrnet/clr_resnet18_culane.yml
+```
+
+### Evaluation
+```shell
+python tools/eval.py -c configs/clrnet/clr_resnet18_culane.yml -o weights=output/clr_resnet18_culane/model_final.pdparams
+```
+
+### Inference
+```shell
+python tools/infer_culane.py -c configs/clrnet/clr_resnet18_culane.yml -o weights=output/clr_resnet18_culane/model_final.pdparams --infer_img=demo/lane00000.jpg
+```
+
+Notice: The inference phase does not support static model graph deploy at present.
+
+## Citations
+```
+@InProceedings{Zheng_2022_CVPR,
+    author    = {Zheng, Tu and Huang, Yifei and Liu, Yang and Tang, Wenjian and Yang, Zheng and Cai, Deng and He, Xiaofei},
+    title     = {CLRNet: Cross Layer Refinement Network for Lane Detection},
+    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
+    month     = {June},
+    year      = {2022},
+    pages     = {898-907}
+}
+```
diff --git a/configs/clrnet/_base_/clrnet_r18_fpn.yml b/configs/clrnet/_base_/clrnet_r18_fpn.yml
@@ -0,0 +1,41 @@
+architecture: CLRNet
+
+CLRNet:
+  backbone: CLRResNet
+  neck: CLRFPN
+  clr_head: CLRHead
+
+CLRResNet:
+  resnet: 'resnet18'
+  pretrained: True
+
+CLRFPN:
+  in_channels: [128,256,512]
+  out_channel: 64
+  extra_stage: 0
+
+CLRHead:
+  prior_feat_channels: 64
+  fc_hidden_dim: 64
+  num_priors: 192
+  num_fc: 2
+  refine_layers: 3
+  sample_points: 36
+  loss: CLRNetLoss
+  conf_threshold: 0.4
+  nms_thres: 0.8
+
+CLRNetLoss:
+  cls_loss_weight : 2.0
+  xyt_loss_weight : 0.2
+  iou_loss_weight : 2.0
+  seg_loss_weight : 1.0
+  refine_layers : 3
+  ignore_label: 255
+  bg_weight: 0.4
+
+# for visualize lane detection results
+sample_y:
+  start: 589
+  end: 230
+  step: -20
diff --git a/configs/clrnet/_base_/clrnet_reader.yml b/configs/clrnet/_base_/clrnet_reader.yml
@@ -0,0 +1,37 @@
+worker_num: 10
+
+img_h: &img_h 320
+img_w: &img_w 800
+ori_img_h: &ori_img_h 590
+ori_img_w: &ori_img_w 1640
+num_points: &num_points 72
+max_lanes: &max_lanes 4
+
+TrainReader:
+  batch_size: 24
+  batch_transforms:
+    - CULaneTrainProcess: {img_h: *img_h, img_w: *img_w}
+    - CULaneDataProcess: {num_points: *num_points, max_lanes: *max_lanes, img_w: *img_w, img_h: *img_h}
+  shuffle: True
+  drop_last: False
+
+
+
+
+EvalReader:
+  batch_size: 24
+  batch_transforms:
+    - CULaneResize: {prob: 1.0, img_h: *img_h, img_w: *img_w}
+    - CULaneDataProcess: {num_points: *num_points, max_lanes: *max_lanes, img_w: *img_w, img_h: *img_h}
+  shuffle: False
+  drop_last: False
+
+
+
+TestReader:
+  batch_size: 24
+  batch_transforms:
+    - CULaneResize: {prob: 1.0, img_h: *img_h, img_w: *img_w}
+    - CULaneDataProcess: {num_points: *num_points, max_lanes: *max_lanes, img_w: *img_w, img_h: *img_h}
+  shuffle: False
+  drop_last: False
diff --git a/configs/clrnet/_base_/optimizer_1x.yml b/configs/clrnet/_base_/optimizer_1x.yml
@@ -0,0 +1,14 @@
+epoch: 15
+snapshot_epoch: 5
+
+LearningRate:
+  base_lr: 0.6e-3
+  schedulers:
+  - !CosineDecay
+    max_epochs: 15
+    use_warmup: False
+
+OptimizerBuilder:
+  regularizer: False
+  optimizer:
+    type: AdamW
diff --git a/configs/clrnet/clrnet_resnet18_culane.yml b/configs/clrnet/clrnet_resnet18_culane.yml
@@ -0,0 +1,9 @@
+_BASE_: [
+  '../datasets/culane.yml',
+  '_base_/clrnet_reader.yml',
+  '_base_/clrnet_r18_fpn.yml',
+  '_base_/optimizer_1x.yml',
+  '../runtime.yml'
+]
+
+weights: output/clr_resnet18_culane/model_final
diff --git a/configs/datasets/culane.yml b/configs/datasets/culane.yml
@@ -0,0 +1,28 @@
+metric: CULaneMetric
+num_classes: 5 # 4 lanes + background
+
+cut_height: &cut_height 270
+dataset_dir: &dataset_dir dataset/culane
+
+TrainDataset:
+  name: CULaneDataSet
+  dataset_dir: *dataset_dir
+  list_path: 'list/train_gt.txt'
+  split: train
+  cut_height: *cut_height
+
+
+EvalDataset:
+  name: CULaneDataSet
+  dataset_dir: *dataset_dir
+  list_path: 'list/test.txt'
+  split: test
+  cut_height: *cut_height
+
+
+TestDataset:
+  name: CULaneDataSet
+  dataset_dir: *dataset_dir
+  list_path: 'list/test.txt'
+  split: test
+  cut_height: *cut_height
diff --git a/demo/lane00000.jpg b/demo/lane00000.jpg