Yolox improve with REPConv/ASFF/TOOD (#154)

* add attention layer and more loss function * add attention layer and various loss functions * add siou loss * add tah,various attention layers, and different loss functions * add asff sim, gsconv * blade utils fit faster * blade optimize for yolox static & fp16 * decode output for yolox control by cfg * add reparameterize_models for export * e2e trt_nms plugin export support and numeric test * split preprocess from end2end+blade, speedup from 17ms->7.2ms Co-authored-by: zouxinyi0625 <zouxinyi.zxy@alibaba-inc.com>
alibaba · Aug 24, 2022 · 9aaa600 · 9aaa600
1 parent a75d41f
commit 9aaa600
Show file tree

Hide file tree

Showing 48 changed files with 3,103 additions and 1,599 deletions.
diff --git a/.github/workflows/citest.yaml b/.github/workflows/citest.yaml
@@ -67,35 +67,36 @@ jobs:
           PYTHONPATH=. python tests/run.py
 
 
-  ut-torch181-blade:
-    # The type of runner that the job will run on
-    runs-on: [unittest-t4]
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v2
-      - name: Run unittest
-        shell: bash
-        run: |
-          set -e
-          UNITTEST_OSS_CONFIG=~/.ossutilconfig.unittest
-          if [ ! -e $UNITTEST_OSS_CONFIG ]; then
-              echo "$UNITTEST_OSS_CONFIG does not exists"
-              exit
-          fi
-
-          export OSS_CONFIG_FILE=$UNITTEST_OSS_CONFIG
-
-          export PYTHONPATH=.
-          export CUDA_HOME=/apsarapangu/disk6/xinyi.zxy/cuda-10.2
-          export LD_LIBRARY_PATH=${CUDA_HOME}/lib64
-          export PATH=${CUDA_HOME}/bin:${PATH}
-          export TEST_DIR="/tmp/easycv_test_${USER}_`date +%s`"
-
-          # do not uncomments, casue faild in Online UT, install requirements by yourself on UT machine
-          # pip install -r requirements.txt
-          #run test
-          export CUDA_VISIBLE_DEVICES=6
-          source ~/workspace/anaconda2/etc/profile.d/conda.sh
-          conda activate torch1.8.1_blade
-          PYTHONPATH=. python tests/predictors/test_detector_blade.py
-          PYTHONPATH=. python tests/apis/test_export_blade.py
+# blade test env will be updated!
+#  ut-torch181-blade:
+#    # The type of runner that the job will run on
+#    runs-on: [unittest-t4]
+#    steps:
+#      - name: Checkout
+#        uses: actions/checkout@v2
+#      - name: Run unittest
+#        shell: bash
+#        run: |
+#          set -e
+#          UNITTEST_OSS_CONFIG=~/.ossutilconfig.unittest
+#          if [ ! -e $UNITTEST_OSS_CONFIG ]; then
+#              echo "$UNITTEST_OSS_CONFIG does not exists"
+#              exit
+#          fi
+#
+#          export OSS_CONFIG_FILE=$UNITTEST_OSS_CONFIG
+#
+#          export PYTHONPATH=.
+#          export CUDA_HOME=/apsarapangu/disk6/xinyi.zxy/cuda-10.2
+#          export LD_LIBRARY_PATH=${CUDA_HOME}/lib64
+#          export PATH=${CUDA_HOME}/bin:${PATH}
+#          export TEST_DIR="/tmp/easycv_test_${USER}_`date +%s`"
+#
+#          # do not uncomments, casue faild in Online UT, install requirements by yourself on UT machine
+#          # pip install -r requirements.txt
+#          #run test
+#          export CUDA_VISIBLE_DEVICES=6
+#          source ~/workspace/anaconda2/etc/profile.d/conda.sh
+#          conda activate torch1.8.1_blade
+#          PYTHONPATH=. python tests/predictors/test_detector_blade.py
+#          PYTHONPATH=. python tests/apis/test_export_blade.py
diff --git a/README.md b/README.md
@@ -22,6 +22,8 @@ English | [简体中文](README_zh-CN.md)
 
 EasyCV is an all-in-one computer vision toolbox based on PyTorch, mainly focus on self-supervised learning, transformer based models, and SOTA CV tasks including image classification, metric-learning, object detection, pose estimation and so on.
 
+[🔥 Latest News] We have released our YOLOX-PAI that reveives SOTA results within 40~50 mAP (less than 1ms). And we also provide a convenient and fast export/predictor api for end2end object detection. To get a quick start of YOLOX-PAI, click [here](docs/source/tutorials/yolox.md)!
+
 ### Major features
 
 - **SOTA SSL Algorithms**

diff --git a/assets/result.jpg b/assets/result.jpg
diff --git a/configs/detection/yolox/yolox_l_8xb8_300e_coco.py b/configs/detection/yolox/yolox_l_8xb8_300e_coco.py
@@ -1,7 +1,7 @@
 _base_ = './yolox_s_8xb16_300e_coco.py'
 
 # model settings
-model = dict(model_type='l')
+model = dict(model_type='l', head=dict(model_type='l', ))
 
 data = dict(imgs_per_gpu=8, workers_per_gpu=4)
 

diff --git a/configs/detection/yolox/yolox_m_8xb16_300e_coco.py b/configs/detection/yolox/yolox_m_8xb16_300e_coco.py
@@ -1,4 +1,4 @@
 _base_ = './yolox_s_8xb16_300e_coco.py'
 
 # model settings
-model = dict(model_type='m')
+model = dict(model_type='m', head=dict(model_type='m', ))
diff --git a/configs/detection/yolox/yolox_nano_8xb16_300e_coco.py b/configs/detection/yolox/yolox_nano_8xb16_300e_coco.py
@@ -1,4 +1,4 @@
 _base_ = './yolox_tiny_8xb16_300e_coco.py'
 
 # model settings
-model = dict(model_type='nano')
+model = dict(model_type='nano', head=dict(model_type='nano', ))
diff --git a/configs/detection/yolox/yolox_s_8xb16_300e_coco.py b/configs/detection/yolox/yolox_s_8xb16_300e_coco.py
@@ -3,10 +3,17 @@
 # model settings s m l x
 model = dict(
     type='YOLOX',
-    num_classes=80,
-    model_type='s',  # s m l x tiny nano
     test_conf=0.01,
-    nms_thre=0.65)
+    nms_thre=0.65,
+    backbone='CSPDarknet',
+    model_type='s',  # s m l x tiny nano
+    head=dict(
+        type='YOLOXHead',
+        model_type='s',
+        obj_loss_type='BCE',
+        reg_loss_type='giou',
+        num_classes=80,
+        decode_in_inference=True))
 
 # s m l x
 img_scale = (640, 640)
@@ -36,6 +43,7 @@
 
 # dataset settings
 data_root = 'data/coco/'
+
 img_norm_cfg = dict(
     mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
 
@@ -82,7 +90,7 @@
             dict(type='LoadAnnotations', with_bbox=True)
         ],
         classes=CLASSES,
-        filter_empty_gt=False,
+        filter_empty_gt=True,
         iscrowd=False),
     pipeline=train_pipeline,
     dynamic_scale=img_scale)
@@ -100,6 +108,7 @@
         ],
         classes=CLASSES,
         filter_empty_gt=False,
+        test_mode=True,
         iscrowd=True),
     pipeline=test_pipeline,
     dynamic_scale=None,
@@ -179,4 +188,4 @@
         # dict(type='WandbLoggerHookV2'),
     ])
 
-export = dict(use_jit=False, export_blade=False, end2end=False)
+export = dict(export_type = 'raw', preprocess_jit = False, batch_size=1, blade_config=dict(enable_fp16=True, fp16_fallback_op_ratio=0.01), use_trt_efficientnms=False)
diff --git a/configs/detection/yolox/yolox_tiny_8xb16_300e_coco.py b/configs/detection/yolox/yolox_tiny_8xb16_300e_coco.py
@@ -1,7 +1,7 @@
 _base_ = './yolox_s_8xb16_300e_coco.py'
 
 # model settings
-model = dict(model_type='tiny')
+model = dict(model_type='tiny', head=dict(model_type='tiny', ))
 
 CLASSES = [
     'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train',

diff --git a/configs/detection/yolox/yolox_x_8xb8_300e_coco.py b/configs/detection/yolox/yolox_x_8xb8_300e_coco.py
@@ -1,7 +1,7 @@
 _base_ = './yolox_s_8xb16_300e_coco.py'
 
 # model settings
-model = dict(model_type='x')
+model = dict(model_type='x', head=dict(model_type='x', ))
 
 data = dict(imgs_per_gpu=8, workers_per_gpu=4)
 

diff --git a/configs/edge_models/yolox_edge.py b/configs/edge_models/yolox_edge.py
@@ -7,7 +7,6 @@
 model = dict(
     stage='EDGE',
     type='YOLOX_EDGE',
-    num_classes=1,
     model_type='customized',
     test_conf=0.01,
     nms_thre=0.65,
@@ -16,14 +15,19 @@
     max_model_params=-1,
     max_model_flops=-1,
     activation='relu',
-)
+    head=dict(
+        type='YOLOXHead',
+        model_type='customized',
+        num_classes=1,
+        reg_loss_type='iou',
+        width=1.0))
 
 # train setting
 samples_per_gpu = 16  # batch size per gpu
 test_samples_per_gpu = 16  # test batch size per gpu
 gpu_num = 2  # gpu number for one worker
 total_epochs = 11  # train epoch
-interval = 5
+interval = 5  # eval interval
 
 # tiny nano without mixup
 img_scale = (256, 256)

diff --git a/docs/source/model_zoo_det.md b/docs/source/model_zoo_det.md
@@ -4,14 +4,17 @@
 
 Pretrained on COCO2017 dataset.
 
-| Algorithm  | Config                                                       | Params                                                 | inference time(V100)<br/>(ms/img)                      | mAP<sup>val<br/><sub>0.5:0.95</sub> | AP<sup>val<br/><sub>50</sub> | Download                                                     |
-| ---------- | ------------------------------------------------------------ | ------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
-| YOLOX-s    | [yolox_s_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_s_8xb16_300e_coco.py) | 9M | 10.7ms | 40.0                   | 58.9          | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_s_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_s_bs16_lr002/log.txt) |
-| YOLOX-m    | [yolox_m_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_m_8xb16_300e_coco.py) | 25M | 12.3ms | 46.3                   | 64.9          | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_m_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_m_bs16_lr002/log.txt) |
-| YOLOX-l    | [yolox_l_8xb8_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_m_8xb8_300e_coco.py) | 54M | 15.5ms | 48.9                  | 67.5        | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_l_bs8_lr001/epoch_290.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_l_bs8_lr001/log.txt) |
-| YOLOX-x    | [yolox_x_8xb8_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_x_8xb8_300e_coco.py) | 99M | 19ms | 50.9                   | 69.2          | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_x_bs8_lr001/epoch_290.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_x_bs8_lr001/log.txt) |
-| YOLOX-tiny | [yolox_tiny_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_tiny_8xb16_300e_coco.py) | 5M | 9.5ms | 31.5                   | 49.2          | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_tiny_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_tiny_bs16_lr002/log.txt) |
-| YOLOX-nano | [yolox_nano_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_tiny_8xb16_300e_coco.py) | 2.2M | 9.4ms | 26.5                   | 42.6          | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_nano_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_nano_bs16_lr002/log.txt) |
+| Algorithm             | Config                                                                                                                                                              | Params | Speed<sup>V100<br/><sub>fp16 b32 </sub> | mAP<sup>val<br/><sub>0.5:0.95</sub> | AP<sup>val<br/><sub>50</sub> | Download                                                                                                                                                                                                                                                                         |
+|-----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------|-----------------------------------------|-------------------------------------|------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| YOLOX-s               | [yolox_s_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_s_8xb16_300e_coco.py)                                         | 9M     | 0.68ms                                  | 40.0                                | 58.9                         | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_s_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_s_bs16_lr002/log.txt)                |
+| PAI-YOLOXs            | [yoloxs_pai_8xb16_300e_coco](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/config/pai_yoloxs.py)                  | 16M    | 0.71ms                                  | 41.4                                | 60.0                         | [model](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/model/pai_yoloxs.pth) - [log](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/log/pai_yoloxs.json)                       |
+| PAI-YOLOXs-ASFF       | [yoloxs_pai_asff_8xb16_300e_coco](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/config/pai_yoloxs_asff.py)        | 21M    | 0.87ms                                  | 42.8                                | 61.8                         | [model](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/model/pai_yoloxs_asff.pth) - [log](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/log/pai_yoloxs_asff.json)             |
+| PAI-YOLOXs-ASFF-TOOD3 | [yoloxs_pai_asff_tood3_8xb16_300e_coco](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/config/pai_yoloxs_tood3.py) | 24M    | 1.15ms                                  | 43.9                                | 62.1                         | [model](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/model/pai_yoloxs_asff_tood3.pth) - [log](http://pai-vision-data-hz.oss-accelerate.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox-pai/log/pai_yoloxs_asff_tood3.json) |
+| YOLOX-m               | [yolox_m_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_m_8xb16_300e_coco.py)                                         | 25M    | 1.52ms                                  | 46.3                                | 64.9                         | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_m_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_m_bs16_lr002/log.txt)                |
+| YOLOX-l               | [yolox_l_8xb8_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_m_8xb8_300e_coco.py)                                           | 54M    | 2.47ms                                  | 48.9                                | 67.5                         | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_l_bs8_lr001/epoch_290.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_l_bs8_lr001/log.txt)                  |
+| YOLOX-x               | [yolox_x_8xb8_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_x_8xb8_300e_coco.py)                                           | 99M    | 4.74ms                                  | 50.9                                | 69.2                         | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_x_bs8_lr001/epoch_290.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_x_bs8_lr001/log.txt)                  |
+| YOLOX-tiny            | [yolox_tiny_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_tiny_8xb16_300e_coco.py)                                   | 5M     | 0.28ms                                  | 31.5                                | 49.2                         | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_tiny_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_tiny_bs16_lr002/log.txt)          |
+| YOLOX-nano            | [yolox_nano_8xb16_300e_coco](https://github.com/alibaba/EasyCV/tree/master/configs/detection/yolox/yolox_tiny_8xb16_300e_coco.py)                                   | 2.2M   | 0.19ms                                  | 26.5                                | 42.6                         | [model](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_nano_bs16_lr002/epoch_300.pth) - [log](http://pai-vision-data-hz.oss-cn-zhangjiakou.aliyuncs.com/EasyCV/modelzoo/detection/yolox/yolox_nano_bs16_lr002/log.txt)          |
 
 ## ViTDet