Torchscript support (open-mmlab#159)

* support torchscript * add nms * add torchscript configs and update deploy process and dump-info * typescript -> torchscript * add torchscript custom extension support * add ts custom ops again * support mmseg unet * [WIP] add optimizer for torchscript (open-mmlab#119) * add passes * add python api * Torchscript optimizer python api (open-mmlab#121) * add passes * add python api * use python api instead of executable * Merge Master, update optimizer (open-mmlab#151) * [Feature] add yolox ncnn (open-mmlab#29) * add yolox ncnn * add ncnn android performance of yolox * add ut * fix lint * fix None bugs for ncnn * test codecov * test codecov * add device * fix yapf * remove if-else for img shape * use channelshuffle optimize * change benchmark after channelshuffle * fix yapf * fix yapf * fuse continuous reshape * fix static shape deploy * fix code * drop pad * only static shape * fix static * fix docstring * Added mask overlay to output image, changed fprintf info messages to … (open-mmlab#55) * Added mask overlay to output image, changed fprintf info messages to stdout * Improved box filtering (filter area/score), make sure roi coordinates stay within bounds * clang-format * Support UNet in mmseg (open-mmlab#77) * Repeatdataset in train has no CLASSES & PALETTE * update result for unet * update docstring for mmdet * remove ppl for unet in docs * fix ort wrap about input type (open-mmlab#81) * Fix memleak (open-mmlab#86) * delete [] * fix build error when enble MMDEPLOY_ACTIVE_LEVEL * fix lint * [Doc] Nano benchmark and tutorial (open-mmlab#71) * add cls benchmark * add nano zh-cn benchmark and en tutorial * add device row * add doc path to index.rst * fix typo * [Fix] fix missing deploy_core (open-mmlab#80) * fix missing deploy_core * mv flag to demo * target link * [Docs] Fix links in Chinese doc (open-mmlab#84) * Fix docs in Chinese link * Fix links * Delete symbolic link and add links to html * delete files * Fix link * [Feature] Add docker files (open-mmlab#67) * add gpu and cpu dockerfile * fix lint * fix cpu docker and remove redundant * use pip instead * add build arg and readme * fix grammar * update readme * add chinese doc for dockerfile and add docker build to build.md * grammar * refine dockerfiles * add FAQs * update Dpplcv_DIR for SDK building * remove mmcls * add sdk demos * fix typo and lint * update FAQs * [Fix]fix check_env (open-mmlab#101) * fix check_env * update * Replace convert_syncbatchnorm in mmseg (open-mmlab#93) * replace convert_syncbatchnorm with revert_sync_batchnorm from mmcv * change logger * [Doc] Update FAQ for TensorRT (open-mmlab#96) * update FAQ * comment * [Docs]: Update doc for openvino installation (open-mmlab#102) * fix docs * fix docs * fix docs * fix mmcv version * fix docs * rm blank line * simplify non batch nms (open-mmlab#99) * [Enhacement] Allow test.py to save evaluation results (open-mmlab#108) * Add log file * Delete debug code * Rename logger * resolve comments * [Enhancement] Support mmocr v0.4+ (open-mmlab#115) * support mmocr v0.4+ * 0.4.0 -> 0.4.1 * fix onnxruntime wrapper for gpu inference (open-mmlab#123) * fix ncnn wrapper for ort-gpu * resolve comment * fix lint * Fix typo (open-mmlab#132) * lock mmcls version (open-mmlab#131) * [Enhancement] upgrade isort in pre-commit config (open-mmlab#141) * [Enhancement] upgrade isort in pre-commit config by refering to mmflow pr open-mmlab#87 * fix lint * remove .isort.cfg and put its known_third_party to setup.cfg * Fix ci for mmocr (open-mmlab#144) * fix mmocr unittests * remove useless * lock mmdet maximum version to 2.20 * pip install -U numpy * Fix capture_output (open-mmlab#125) Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com> Co-authored-by: Johannes L <tehkillerbee@users.noreply.github.com> Co-authored-by: RunningLeon <mnsheng@yeah.net> Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com> Co-authored-by: lvhan028 <lvhan_028@163.com> Co-authored-by: AllentDan <41138331+AllentDan@users.noreply.github.com> Co-authored-by: Yifan Zhou <singlezombie@163.com> Co-authored-by: 杨培文 (Yang Peiwen) <915505626@qq.com> Co-authored-by: Semyon Bevzyuk <semen.bevzuk@gmail.com> * configs for all tasks * use torchvision roi align * remote unnecessary code * fix ut * fix ut * export * det dynamic * det dynamic * add ut * fix ut * add ut and docs * fix ut * skip torchscript ut if no ops available * add torchscript option to build.md * update benchmark and resolve comments * resolve conflicts * rename configs * fix mrcnn cuda test * remove useless * add version requirements to docs and comments to codes * enable empty image exporting for torchscript and accelerate ORT inference for MRCNN * rebase * update example for torchscript.md * update FAQs for torchscript.md * resolve comments * only use torchvision roi_align for torchscript * fix ut * use torchvision roi align when pool model is avg * resolve comments Co-authored-by: grimoire <streetyao@live.com> Co-authored-by: grimoire <yaoqian@sensetime.com> Co-authored-by: hanrui1sensetime <83800577+hanrui1sensetime@users.noreply.github.com> Co-authored-by: Johannes L <tehkillerbee@users.noreply.github.com> Co-authored-by: RunningLeon <mnsheng@yeah.net> Co-authored-by: VVsssssk <88368822+VVsssssk@users.noreply.github.com> Co-authored-by: lvhan028 <lvhan_028@163.com> Co-authored-by: Yifan Zhou <singlezombie@163.com> Co-authored-by: 杨培文 (Yang Peiwen) <915505626@qq.com> Co-authored-by: Semyon Bevzyuk <semen.bevzuk@gmail.com>
irexyc · Mar 30, 2022 · f060416 · f060416
1 parent 1d8a67b
commit f060416
Show file tree

Hide file tree

Showing 6 changed files with 231 additions and 77 deletions.
diff --git a/docs/en/build.md b/docs/en/build.md
@@ -32,9 +32,151 @@ Note:
     git submodule update --init --recursive
     ```
 
-## Build MMDeploy
-Please visit the following links to find out how to build MMDeploy according to the target platform.
-- [Linux-x86_64](build/linux.md)
-- [Windows](build/windows.md)
-- [Android-aarch64](build/android.md)
-- [NVIDIA Jetson](tutorials/how_to_install_mmdeploy_on_jetsons.md)
+
+
+### Build backend support
+
+Build the inference engine extension libraries you need.
+
+- [ONNX Runtime](backends/onnxruntime.md)
+- [TensorRT](backends/tensorrt.md)
+- [ncnn](backends/ncnn.md)
+- [pplnn](backends/pplnn.md)
+- [OpenVINO](backends/openvino.md)
+- [TorchScript](backends/torchscript.md)
+
+### Install mmdeploy
+
+```bash
+cd ${MMDEPLOY_DIR} # To mmdeploy root directory
+pip install -e .
+```
+
+**Note**
+
+- Some dependencies are optional. Simply running `pip install -e .` will only install the minimum runtime requirements.
+To use optional dependencies, install them manually with `pip install -r requirements/optional.txt` or specify desired extras when calling `pip` (e.g. `pip install -e . [optional]`).
+Valid keys for the extras field are: `all`, `tests`, `build`, `optional`.
+
+### Build SDK
+
+Readers can skip this chapter if you are only interested in model converter.
+
+#### Dependencies
+
+Currently, SDK is tested on Linux x86-64, more platforms will be added in the future. The following packages are required to build MMDeploy SDK.
+
+Each package's installation command is given based on Ubuntu 18.04.
+
+- OpenCV 3+
+
+  ```bash
+  sudo apt-get install libopencv-dev
+  ```
+
+- spdlog 0.16+
+
+  ``` bash
+  sudo apt-get install libspdlog-dev
+  ```
+
+  On Ubuntu 16.04, please use the following command
+  ```bash
+  wget http://archive.ubuntu.com/ubuntu/pool/universe/s/spdlog/libspdlog-dev_0.16.3-1_amd64.deb
+  sudo dpkg -i libspdlog-dev_0.16.3-1_amd64.deb
+  ```
+
+  You can also build spdlog from its source to enjoy its latest features. But be sure to add **`-fPIC`** compilation flags at first.
+
+- pplcv
+
+  A high-performance image processing library of openPPL supporting x86 and cuda platforms.</br>
+  It is **OPTIONAL** which only be needed if `cuda` platform is required.
+
+  Using v0.6.1, since latest updates have broughtup breaking changes
+  ```Bash
+  wget https://github.com/openppl-public/ppl.cv/archive/refs/tags/v0.6.1.zip
+  unzip v0.6.1.zip && mv ppl.cv-0.6.1 ppl.cv
+  cd ppl.cv
+  ./build.sh cuda
+  ```
+
+- backend engines
+
+  SDK uses the same backends as model converter does. Please follow [build backend](#build-backend-support) guide to install your interested backend.
+
+#### Set Build Option
+
+- Turn on SDK build switch
+
+  `-DMMDEPLOY_BUILD_SDK=ON`
+
+
+- Enabling Devices
+
+   By default, only CPU device is included in the target devices. You can enable device support for other devices by
+   passing a semicolon separated list of device names to `MMDEPLOY_TARGET_DEVICES` variable, e.g. `-DMMDEPLOY_TARGET_DEVICES="cpu;cuda"`. </br>
+   Currently, the following devices are supported.
+
+   | device |  name | path setter |
+   |--------|-------|-------------|
+   |  Host  |  cpu  |    N/A      |
+   |  CUDA  |  cuda | CUDA_TOOLKIT_ROOT_DIR & pplcv_DIR |
+
+   If you have multiple CUDA versions installed on your system, you will need to pass `CUDA_TOOLKIT_ROOT_DIR` to cmake to specify the version. </br>
+   Meanwhile, `pplcv_DIR` has to be provided in order to build image processing operators on cuda platform.
+
+
+- Enabling inference engines
+
+   **By default, no target inference engines are set**, since it's highly dependent on the use case.
+   `MMDEPLOY_TARGET_BACKENDS` must be set to a semicolon separated list of inference engine names,
+   e.g. `-DMMDEPLOY_TARGET_BACKENDS="trt;ort;pplnn;ncnn;openvino"`
+   A path to the inference engine library is also needed. The following backends are currently supported
+
+   |   library   |  name    |   path setter   |
+   |-------------|----------|-----------------|
+   | PPL.nn      | pplnn    | pplnn_DIR       |
+   | ncnn        | ncnn     | ncnn_DIR        |
+   | ONNXRuntime | ort      | ONNXRUNTIME_DIR |
+   | TensorRT    | trt      | TENSORRT_DIR & CUDNN_DIR |
+   | OpenVINO    | openvino | InferenceEngine_DIR |
+
+- Enabling codebase's postprocess components
+
+  `MMDEPLOY_CODEBASES` MUST be specified by a semicolon separated list of codebase names.
+  The currently supported codebases are 'mmcls', 'mmdet', 'mmedit', 'mmseg', 'mmocr'.
+  Instead of listing them one by one in `MMDEPLOY_CODEBASES`, user can also pass `all` to enable all of them, i.e.,
+  `-DMMDEPLOY_CODEBASES=all`
+
+
+- Put it all together
+
+  The following is a recipe for building MMDeploy SDK with cpu device and ONNXRuntime support
+  ```Bash
+  mkdir build && cd build
+  cmake .. \
+      -DMMDEPLOY_BUILD_SDK=ON \
+      -DCMAKE_CXX_COMPILER=g++-7 \
+      -DONNXRUNTIME_DIR=/path/to/onnxruntime \
+      -DMMDEPLOY_TARGET_DEVICES=cpu \
+      -DMMDEPLOY_TARGET_BACKENDS=ort \
+      -DMMDEPLOY_CODEBASES=all
+  cmake --build . -- -j$(nproc) && cmake --install .
+  ```
+
+  Here is another example to build MMDeploy SDK with cuda device and TensorRT backend
+
+  ```Bash
+   mkdir build && cd build
+   cmake .. \
+     -DMMDEPLOY_BUILD_SDK=ON \
+     -DCMAKE_CXX_COMPILER=g++-7 \
+     -Dpplcv_DIR=/path/to/ppl.cv/cuda-build/install/lib/cmake/ppl \
+     -DTENSORRT_DIR=/path/to/tensorrt \
+     -DCUDNN_DIR=/path/to/cudnn \
+     -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
+     -DMMDEPLOY_TARGET_BACKENDS=trt \
+     -DMMDEPLOY_CODEBASES=all
+   cmake --build . -- -j$(nproc) && cmake --install .
+  ```