【Hackathon 5th No.68】轻量语义分割网络PIDNet

configs/pidnet/README.md

@@ -0,0 +1,37 @@
+# PIDNet: A Real-time Semantic Segmentation Network Inspired from PID Controller
+
+## Reference
+
+> Xu, Jiacong, Zixiang Xiong, and Shankar P. Bhattacharyya. "PIDNet: A Real-Time Semantic Segmentation Network Inspired by PID Controllers." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.
+
+## Performance
+
+### Cityscapes
+
+| Model | Backbone | Resolution | Training Iters | mIoU | mIoU (flip) | mIoU (ms+flip) | Links |
+|-|-|-|-|-|-|-|-|
+|PIDNet|PIDNet-Small |1024x1024|120000|78.48%|79.02%|79.68%|[model](https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_small_cityscapes_1024x1024_120k/model.pdparams) \| [log](https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_small_cityscapes_1024x1024_120k/pidnet_small.log) \| [vdl](https://paddlepaddle.org.cn/paddle/visualdl/service/app?id=57dda9c34cd06a4b2996118df03583c9)|
+|PIDNet|PIDNet_Medium|1024x1024|120000| | | |
+|PIDNet|PIDNet-Large |1024x1024|120000| | | |
+
+
+#### official weight
+
+| Model | Backbone | Resolution | Training Iters | mIoU | mIoU (flip) | mIoU (ms+flip) | Links |
+|-|-|-|-|-|-|-|-|
+|PIDNet|PIDNet-Small |1024x1024|120000|78.74%|79.53%|80.28%|[model](https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_small_2xb6-120k_1024x1024-cityscapes.pdparams)|
+|PIDNet|PIDNet_Medium|1024x1024|120000|80.22%|81.07%|81.50%|[model](https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_medium_2xb6-120k_1024x1024-cityscapes.pdparams)|
+|PIDNet|PIDNet-Large |1024x1024|120000|80.89%|81.41%|81.92%|[model](https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_large_2xb6-120k_1024x1024-cityscapes.pdparams)|
+
+
+#### NOTE:
+
+The `weight` in the training parameters is dataset dependent, if you are using another dataset, modify the weight please.
+
+``` yaml
+# follow the OCNet, compute the weights by 1/log(pixel_count)
+# see https://github.com/openseg-group/OCNet.pytorch/issues/14
+weight: &weight [0.8373, 0.9180, 0.8660, 1.0345, 1.0166, 0.9969, 0.9754,
+                 1.0489, 0.8786, 1.0023, 0.9539, 0.9843, 1.1116, 0.9037,
+                 1.0865, 1.0955, 1.0865, 1.1529, 1.0507]
+```

configs/pidnet/pidnet_large_cityscapes_1024x1024_120k.yml

@@ -0,0 +1,7 @@
+_base_: 'pidnet_small_cityscapes_1024x1024_120k.yml'
+
+model:
+  head_channels: 256
+  backbone:
+    type: PIDNet_Large
+  pretrained: https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_large_imagenet1k.pdparams

configs/pidnet/pidnet_medium_cityscapes_1024x1024_120k.yml

@@ -0,0 +1,6 @@
+_base_: 'pidnet_small_cityscapes_1024x1024_120k.yml'
+
+model:
+  backbone:
+    type: PIDNet_Medium
+  pretrained: https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_medium_imagenet1k.pdparams

configs/pidnet/pidnet_small_cityscapes_1024x1024_120k.yml

@@ -0,0 +1,62 @@
+_base_: '../_base_/cityscapes_1024x1024.yml'
+
+batch_size: 6  # total batch size is 12
+iters: 120000  # 484 epochs
+
+mean: &mean [0.485, 0.456, 0.406]
+std: &std [0.229, 0.224, 0.225]
+
+# follow the OCNet, compute the weights by 1/log(pixel_count)
+# see https://github.com/openseg-group/OCNet.pytorch/issues/14
+weight: &weight [0.8373, 0.9180, 0.8660, 1.0345, 1.0166, 0.9969, 0.9754,
+                 1.0489, 0.8786, 1.0023, 0.9539, 0.9843, 1.1116, 0.9037,
+                 1.0865, 1.0955, 1.0865, 1.1529, 1.0507]
+
+model:
+  type: PIDNet
+  num_classes: 19
+  head_channels: 128
+  backbone:
+    type: PIDNet_Small
+  pretrained: https://paddleseg.bj.bcebos.com/dygraph/pidnet/pidnet_small_imagenet1k.pdparams
+
+train_dataset:
+  transforms:
+    - type: AddEdgeLabel
+    - type: ResizeStepScaling
+      min_scale_factor: 0.5
+      max_scale_factor: 2.1
+      scale_step_size: 0.1
+    - type: RandomPaddingCrop
+      crop_size: [1024, 1024]
+    - type: RandomHorizontalFlip
+    - type: Normalize
+      mean: *mean
+      std: *std
+
+val_dataset:
+  transforms:
+    - type: Normalize
+      mean: *mean
+      std: *std
+
+optimizer:
+  weight_decay: 0.0005
+
+loss:
+  types:
+    - type: CrossEntropyLoss
+      weight: *weight
+      avg_non_ignore: False
+    - type: OhemCrossEntropyLoss
+      weight: *weight
+      min_kept: 131072
+      thresh: 0.9
+    - type: BCELoss
+      weight: dynamic
+      edge_label: True
+    - type: OhemCrossEntropyLoss
+      weight: *weight
+      min_kept: 131072
+      thresh: 0.9
+  coef: [0.4, 1.0, 10.0, 1.0]

paddleseg/datasets/dataset.py

@@ -165,6 +165,11 @@ def __getitem__(self, idx):
                 edge_mask = F.mask_to_binary_edge(
                     data['label'], radius=2, num_classes=self.num_classes)
                 data['edge'] = edge_mask
+            elif 'edge' in data:  # for AddEdgeLabel
+                # F.mask_to_binary_edge is so slow
+                # AddEdgeLabel will faster
+                # But offline generation of edges might be better
+                data['edge'][data['edge'] == self.ignore_index] = 0
         return data
 
     def __len__(self):

paddleseg/models/__init__.py

@@ -75,3 +75,4 @@
 from .segnext import SegNeXt
 from .knet import KNet
 from .pp_mobileseg import PPMobileSeg
+from .pidnet import PIDNet

paddleseg/models/backbones/__init__.py

@@ -33,3 +33,4 @@
 from .vit_adapter import *
 from .mscan import *
 from .seaformer import *
+from .pidnet import *