Develop

EISeg/docs/remote_sensing.md

@@ -4,7 +4,7 @@
 
 ## 1 环境配置
 
-EISeg中对遥感数据的支持来自GDAL/OGR，GDAL是一个在X/MIT许可协议下的开源栅格空间数据转换库，OGR与其功能类似但主要提供对矢量数据的支持。
+EISeg中对遥感数据的支持来自GDAL/OGR，GDAL是一个在X/MIT许可协议下的开源栅格空间数据转换库，OGR与其功能类似但主要提供对矢量数据的支持。同时需要安装rasterio。
 
 ### 1.1 依赖安装
 
@@ -31,7 +31,13 @@ Mac用户建议利用conda安装，如下：
 ```shell script
 conda install gdal
 ```
+#### 1.1.3 rasterio 安装
 
+建议用户利用conda安装，如下
+
+```shell script
+conda install rasterio
+```
 
 ## 2 功能介绍
 

README.md

@@ -8,6 +8,7 @@ English | [简体中文](README_CN.md)
 
 **A High-Efficient Development Toolkit for Image Segmentation based on [PaddlePaddle](https://github.com/paddlepaddle/paddle).**
 
+[![Build Status](https://travis-ci.org/PaddlePaddle/PaddleSeg.svg?branch=release/2.1)](https://travis-ci.org/PaddlePaddle/PaddleSeg)
 [![License](https://img.shields.io/badge/license-Apache%202-blue.svg)](LICENSE)
 [![Version](https://img.shields.io/github/release/PaddlePaddle/PaddleSeg.svg)](https://github.com/PaddlePaddle/PaddleSeg/releases)
 ![python version](https://img.shields.io/badge/python-3.6+-orange.svg)
@@ -19,9 +20,21 @@ English | [简体中文](README_CN.md)
 
 ## News <img src="./docs/images/seg_news_icon.png" width="40"/>
 
-* [2022-01-04] We published a paper on portrait segmentation named [PP-HumanSeg](./contrib/PP-HumanSeg/paper.md), and released Semantic Connectivity-aware Learning (SCL) framework and a Large-Scale Teleconferencing Video Dataset.
-* [2021-10-11] We released PaddleSeg v2.3 with the improved interactive segmentation tool ([EISeg v0.3](./EISeg)), two [matting]((./contrib/Matting)) algorithms, and segmentation model compression.
-* For more information please refer to [Release Note](https://github.com/PaddlePaddle/PaddleSeg/releases).
+<ul class="nobull">
+ <li>[2022-01-20] PaddleSeg v2.4 is released! More details in <a href="https://github.com/PaddlePaddle/PaddleSeg/releases">Release Notes</a>.</li>
+ <ul>
+        <li>Upgrade the interactive annotation tool <a href="./EISeg">EISeg v0.4</a> with faster inference speed, supporting new areas in remote sensing and medical segmentation.</li>
+        <li>Publish the <a href="./contrib/PP-HumanSeg/paper.md">PP-HumanSeg</a> paper, and released Semantic Connectivity-aware Learning framework and a Large-Scale Teleconferencing Video Dataset.</li>
+        <li>Release <a href="./contrib/Matting">PP-HumanMatting</a> for extremely fine-grained human segmentation, with an Android tutorial.</li>
+        <li>Add 9 semantic segmentatioin models, including a new transformer model <a href="configs/segmenter">SegMenter</a>.</li>
+        <li>Provide a comprehensive <a href="docs/model_zoo_overview.md">performance table</a> for all segmentation models, which helps to choose a better model on the accuracy and the speed.</li>
+    </ul> 
+
+
+ <li>[2021-10-11] We released PaddleSeg v2.3 with the improved interactive segmentation tool EISeg v0.3, two matting algorithms, and segmentation model compression.</li>
+ <li>[2021-09-20] We published an interactive segmentation <a href="https://arxiv.org/abs/2109.09406">paper</a> with SOTA performance on several well-known datasets.</li>
+</ul>
+
 
 ## Introduction
 
@@ -107,6 +120,8 @@ PaddleSeg is an end-to-end high-efficent development toolkit for image segmentat
             <li>ESPNetV1</li>
             <li>ENCNet</li>
             <li>PFPNNet</li>
+            <li>FastFCN</li>
+            <li>BiSeNetV1</li>
             <li>SETR</li>
             <li>MLA Transformer</li>
             <li>SegFormer</li>
@@ -321,7 +336,7 @@ PaddleSeg is released under the [Apache 2.0 license](LICENSE).
 * Thanks [yazheng0307](https://github.com/yazheng0307) (LIU Zheng) for contributing quick-start document.
 * Thanks [CuberrChen](https://github.com/CuberrChen) for contributing STDC(rethink BiSeNet), PointRend and DetailAggregateLoss.
 * Thanks [stuartchen1949](https://github.com/stuartchen1949) for contributing SegNet.
-* Thanks [justld](https://github.com/justld) (Lang Du) for contributing ESPNetV2, DMNet, ENCNet, HRNet_W48_Contrast, SECrossEntropyLoss and PixelContrastCrossEntropyLoss.
+* Thanks [justld](https://github.com/justld) (Lang Du) for contributing ESPNetV2, DMNet, ENCNet, HRNet_W48_Contrast, FastFCN, BiSeNetV1, SECrossEntropyLoss and PixelContrastCrossEntropyLoss.
 * Thanks [Herman-Hu-saber](https://github.com/Herman-Hu-saber) (Hu Huiming) for contributing ESPNetV2.
 * Thanks [zhangjin12138](https://github.com/zhangjin12138) for contributing RandomCenterCrop.
 * Thanks [simuler](https://github.com/simuler) for contributing ESPNetV1.

README_CN.md

@@ -19,10 +19,14 @@
 
 ## 最新动态 <img src="./docs/images/seg_news_icon.png" width="40"/>
 
-* [2022-01-04] PaddleSeg团队发表人像分割论文[PP-HumanSeg](./contrib/PP-HumanSeg/paper.md)，并开源连通性学习（SCL）方法和大规模视频会议数据集。
-* [2021-10-11] PaddleSeg 2.3版本发布交互式分割工具([EISeg v0.3](./EISeg)), 开源两种 [Matting](./contrib/Matting) 算法, 以及分割高阶功能[模型蒸馏](./slim/distill)和[模型量化](./slim/quant)方案。
+* [2022-01-20] PaddleSeg 2.4版本发布！详细发版信息请参考[Release Note](https://github.com/PaddlePaddle/PaddleSeg/releases)。
+  * 发布智能标注工具[EISeg v0.4](./EISeg)版，采用静态图预测加速交互过程，并支持遥感和医疗领域智能标注。
+  * 发表人像分割论文[PP-HumanSeg](./contrib/PP-HumanSeg/paper.md)，并开源连通性学习（SCL）方法和大规模视频会议数据集。
+  * 开源发丝级人像抠图Matting算法[PP-HumanMatting](./contrib/Matting)，并提供安卓端教程。
+  * 新增9个经典分割模型, 包括Transformer系列模型[SegMenter](configs/segmenter)，总模型数达到40个。
+  * 提供分割模型[性能对比图](docs/model_zoo_overview_cn.md)，全面展示分割模型性能，方便模型选型。
+* [2021-10-11] PaddleSeg 2.3版本发布交互式分割工具EISeg v0.3, 开源两种[Matting](./contrib/Matting)算法, 以及分割高阶功能[模型蒸馏](./slim/distill)和[模型量化](./slim/quant)方案。
 * [2021-09-20] PaddleSeg团队发表交互式分割论文[EdgeFlow](https://arxiv.org/abs/2109.09406)，已在多个数据集实现SOTA性能。
-* 详细发版信息请参考[Release Note](https://github.com/PaddlePaddle/PaddleSeg/releases).
 
 ## 简介
 PaddleSeg是基于飞桨PaddlePaddle开发的端到端图像分割开发套件，涵盖了**高精度**和**轻量级**等不同方向的大量高质量分割模型。通过模块化的设计，提供了**配置化驱动**和**API调用**两种应用方式，帮助开发者更便捷地完成从训练到部署的全流程图像分割应用。
@@ -115,6 +119,8 @@ PaddleSeg是基于飞桨PaddlePaddle开发的端到端图像分割开发套件
             <li>ESPNetV1</li>
             <li>ENCNet</li>
             <li>PFPNNet</li>
+            <li>FastFCN</li>
+            <li>BiSeNetV1</li>
             <li>SETR</li>
             <li>MLA Transformer</li>
             <li>SegFormer</li>
@@ -324,7 +330,7 @@ PaddleSeg是基于飞桨PaddlePaddle开发的端到端图像分割开发套件
 - 非常感谢[yazheng0307](https://github.com/yazheng0307) (刘正)贡献快速开始教程文档。
 - 非常感谢[CuberrChen](https://github.com/CuberrChen)贡献STDC (rethink BiSeNet) PointRend，和 Detail Aggregate损失函数。
 - 非常感谢[stuartchen1949](https://github.com/stuartchen1949)贡献 SegNet。
-- 非常感谢[justld](https://github.com/justld)(郎督)贡献 ESPNetV2, DMNet, ENCNet, HRNet_W48_Contrast, SECrossEntropyLoss 和PixelContrastCrossEntropyLoss。
+- 非常感谢[justld](https://github.com/justld)(郎督)贡献 ESPNetV2, DMNet, ENCNet, HRNet_W48_Contrast, BiSeNetV1, FastFCN, SECrossEntropyLoss 和PixelContrastCrossEntropyLoss。
 - 非常感谢[Herman-Hu-saber](https://github.com/Herman-Hu-saber)(胡慧明)参与贡献 ESPNetV2。
 - 非常感谢[zhangjin12138](https://github.com/zhangjin12138)贡献数据增强方法 RandomCenterCrop。
 - 非常感谢[simuler](https://github.com/simuler) 贡献 ESPNetV1。

configs/enet/README.md

@@ -0,0 +1,14 @@
+# ENet: A Deep Neural Network Architecture forReal-Time Semantic Segmentation
+
+## Reference
+> Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello. "ENet: A Deep Neural Network Architecture for
+Real-Time Semantic Segmentation." arXiv preprint arXiv:1606.02147(2016).
+
+## Performance
+
+### Cityscapes
+
+| Model | Backbone | Resolution | Training Iters | mIoU | mIoU (flip) | mIoU (ms+flip) | Links |
+|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
+|ENet|-|1024x512|80000|67.42%|68.11%|67.99%|[model](https://bj.bcebos.com/paddleseg/dygraph/cityscapes/bisenetv1_cityscapes_1024x512_160k/model.pdparams)\|[log](https://bj.bcebos.com/paddleseg/dygraph/cityscapes/bisenetv1_cityscapes_1024x512_160k/train.log)\|[vdl](https://paddlepaddle.org.cn/paddle/visualdl/service/app?id=5d57386cdfcdb6a6bcb5135af134a0f2)|
+|

configs/enet/enet_cityscapes_1024x512_adam_0.002_80k.yml

@@ -0,0 +1,31 @@
+_base_: '../_base_/cityscapes.yml'
+batch_size: 8
+
+train_dataset:
+  type: Cityscapes
+  dataset_root: data/cityscapes
+  transforms:
+    - type: RandomPaddingCrop
+      crop_size: [1024, 512]
+    - type: RandomDistort
+      brightness_range: 0.4
+      contrast_range: 0.4
+      saturation_range: 0.4
+    - type: Normalize
+  mode: train
+
+model:
+  type: ENet
+  num_classes: 19
+  pretrained: Null
+
+optimizer:
+  _inherited_: False
+  type: adam
+  weight_decay: 0.0002
+
+lr_scheduler:
+  end_lr: 0
+  learning_rate: 0.001
+  power: 0.9
+  type: PolynomialDecay

contrib/LaneSeg/README.md

@@ -227,6 +227,6 @@ python deploy/python/infer.py --help
 ```
 
 #### Paddle Inference（C++）
-reference [Paddle Inference tutorial](../../deploy/cpp/)
+reference [Paddle Inference tutorial](./deploy/cpp/README.md)
 
 the C++ sources files of the project is in LaneSeg/deploy/cpp

contrib/LaneSeg/README_CN.md

@@ -230,6 +230,6 @@ python deploy/python/infer.py --help
 ```
 
 #### Paddle Inference部署（C++）
-参见[Paddle Inference部署教程](../../deploy/cpp/)
+参见[Paddle Inference部署教程](./deploy/cpp/README_cn.md)
 
 本项目使用的C++源文件在LaneSeg/deploy/cpp目录下

contrib/LaneSeg/deploy/cpp/CMakeLists.txt

@@ -1,16 +1,12 @@
 cmake_minimum_required(VERSION 3.0)
 project(cpp_inference_demo CXX C)
 
-option(WITH_MKL        "Compile demo with MKL/OpenBlas support, default use MKL."       OFF)
+option(WITH_MKL        "Compile demo with MKL/OpenBlas support, default use MKL."       ON)
 option(WITH_GPU        "Compile demo with GPU/CPU, default use CPU."                    OFF)
 option(WITH_STATIC_LIB "Compile demo with static/shared library, default use static."   ON)
 option(USE_TENSORRT "Compile demo with TensorRT."   OFF)
 option(WITH_ROCM "Compile demo with rocm." OFF)
 
-
-set(PADDLE_LIB ${CMAKE_SOURCE_DIR}/paddle)
-set(DEMO_NAME test_seg)
-
 if(NOT WITH_STATIC_LIB)
   add_definitions("-DPADDLE_WITH_SHARED_LIB")
 else()
@@ -150,7 +146,7 @@ else()
 endif()
 
 if (NOT WIN32)
-  set(EXTERNAL_LIB "-ldl -lpthread")
+  set(EXTERNAL_LIB "-lrt -ldl -lpthread")
   set(DEPS ${DEPS}
       ${MATH_LIB} ${MKLDNN_LIB}
       glog gflags protobuf xxhash cryptopp

contrib/LaneSeg/deploy/cpp/README.md

@@ -0,0 +1,46 @@
+English | [简体中文](README_CN.md)
+
+## Deploy the PaddleSeg model using Paddle Inference C++
+
+
+### 1、Install
+
+- Paddle Inference C++
+
+- OpenCV
+
+- Yaml
+
+ More install informations，please refer to [Tutorial](../../../../docs/deployment/inference/cpp_inference.md)。
+
+### 2、Models and Pictures
+
+ - Downdload model
+
+   Enter to `LaneSeg/` directory, and execute commands as follows:
+```shell
+   mkdir output # if not exists
+   wget -P output https://paddleseg.bj.bcebos.com/lane_seg/bisenet/model.pdparams
+```
+ - Export Model
+
+```shell
+   python export.py \
+    --config configs/bisenetV2_tusimple_640x368_300k.yml \
+    --model_path output/model.pdparams \
+    --save_dir output/export
+```  
+
+ - Using the image `data/test_images/3.jpg`
+
+### 3、Compile and execute
+
+Enter to the `LaneSeg/deploy/cpp`
+
+Execute `sh run_seg_cpu.sh`, it will compile and then perform prediction on X86 CPU.
+
+Execute `sh run_seg_gpu.sh`, it will compile and then perform prediction on Nvidia GPU.
+
+The result will be saved in the`out_img_seg.jpg` and `out_image_points.jpg` images
+
+- Note：For the path of the model and image, you can change the files `run_seg_cpu.sh` and `run_seg_gpu.sh` as needed

contrib/LaneSeg/deploy/cpp/README_CN.md

@@ -0,0 +1,44 @@
+简体中文 | [English](README.md)
+
+## 使用Paddle Inference C++部署PaddleSeg模型
+
+### 1、安装
+
+- Paddle Inference C++
+
+- OpenCV
+
+- Yaml
+
+ 更多的安装信息，请参考[教程](../../../../docs/deployment/inference/cpp_inference_cn.md)。
+
+### 2、模型和图片
+
+ - 下载模型
+
+ 进入`LaneSeg/`目录下，执行如下命令:
+```shell
+   mkdir output # if not exists
+   wget -P output https://paddleseg.bj.bcebos.com/lane_seg/bisenet/model.pdparams
+```
+ - 导出模型
+```shell
+   python export.py \
+    --config configs/bisenetV2_tusimple_640x368_300k.yml \
+    --model_path output/model.pdparams \
+    --save_dir output/export
+```  
+
+ - 图片使用 `data/test_images/3.jpg`
+
+### 3、编译、执行
+
+进入目录`LaneSeg/deploy/cpp`
+
+执行`sh run_seg_cpu.sh`，会进行编译，然后在X86 CPU上执行预测。
+
+执行`sh run_seg_gpu.sh`，会进行编译，然后在Nvidia GPU上执行预测。
+
+结果会保存在当前目录的`out_img_seg.jpg`和`out_image_points.jpg`图片。
+
+- 注意：对于模型和图片的路径，可以按需要对文件`run_seg_cpu.sh`和`run_seg_gpu.sh`进行修改。

contrib/LaneSeg/deploy/cpp/run_seg_cpu.sh

@@ -0,0 +1,35 @@
+#!/bin/bash
+set +x
+set -e
+
+WITH_MKL=ON
+WITH_GPU=OFF
+USE_TENSORRT=OFF
+DEMO_NAME=test_seg
+
+work_path=$(dirname $(readlink -f $0))
+LIB_DIR="${work_path}/paddle_inference"
+
+# compile
+mkdir -p build
+cd build
+rm -rf *
+
+cmake .. \
+  -DDEMO_NAME=${DEMO_NAME} \
+  -DWITH_MKL=${WITH_MKL} \
+  -DWITH_GPU=${WITH_GPU} \
+  -DUSE_TENSORRT=${USE_TENSORRT} \
+  -DWITH_STATIC_LIB=OFF \
+  -DPADDLE_LIB=${LIB_DIR}
+
+make -j
+
+# run
+cd ..
+# change model_dir and img_path according to your needs
+./build/test_seg \
+    --model_dir=../../output/export/ \
+    --img_path=../../data/test_images/3.jpg \
+    --use_cpu=true \
+    --use_mkldnn=true

contrib/LaneSeg/deploy/cpp/run_seg_gpu.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+set +x
+set -e
+
+WITH_MKL=ON
+WITH_GPU=ON
+USE_TENSORRT=OFF
+DEMO_NAME=test_seg
+
+work_path=$(dirname $(readlink -f $0))
+LIB_DIR="${work_path}/paddle_inference"
+
+# compile
+mkdir -p build
+cd build
+rm -rf *
+
+cmake .. \
+  -DDEMO_NAME=${DEMO_NAME} \
+  -DWITH_MKL=${WITH_MKL} \
+  -DWITH_GPU=${WITH_GPU} \
+  -DUSE_TENSORRT=${USE_TENSORRT} \
+  -DWITH_STATIC_LIB=OFF \
+  -DPADDLE_LIB=${LIB_DIR}
+
+make -j
+
+# run
+cd ..
+# change model_dir and img_path according to your needs
+./build/test_seg \
+    --model_dir=../../output/export/ \
+    --img_path=../../data/test_images/3.jpg \
+    --use_cpu=false