ONNC 1.3.0 Release

Release Note

ONNC framework

[New feature] ONNC supports new operators Clip, Max, Min, ReduceMean, and PRelu.

C Backend

[New feature] ONNC can compile models into C files.
[New feature] ONNC provides a library containing function implementation for 116 neural network operators defined in ONNX rel-1.3.0 specification.
[New feature] The ONNC library can call Intel MKLDNN library for accelerating the computation of convolution and Gemm (matrix multiplication) on Intel CPU.

Supported ONNX Operators

• Support
  • Add
  • AveragePool
  • BatchNormalization
  • Concat
  • Conv
  • Clip new
  • Gemm
  • GlobalAveragePool
  • Identity
  • LRN
  • Max new
  • MaxPool
  • Min new
  • Mul
  • PRelu new
  • Relu
  • ReduceMean new
  • Reshape
  • Softmax
  • Sum
  • Transpose (use in ShuffleNet)
  • Unsqueeze

Supported ONNX Models

Model	NVDLA nv_full Backend	C Backend
bvlc_alexnet	O	O
bvlc_googlenet	O	O
bvlc_reference_caffenet	O	O
bvlc_reference_rcnn_ilsvrc13	O	O
densenet121	O	O
inception_v1	O	O
inception_v2	O	O
resnet50	O	O
shufflenet	O	O
squeezenet	O	O
vgg19	O	O
zfnet512	O	O
pytorch-alexnet	O	O
pytorch-densenet121	O	O
pytorch-densenet161	O	O
pytorch-densenet169	O	O
pytorch-densenet201	O	O
pytorch-googlenet	O	O
pytorch-inception_v3	O	O
pytorch-mnasnet0_5	X	O
pytorch-mnasnet1_0	X	O
pytorch-mobilenet_v2	X	O
pytorch-resnet101	O	O
pytorch-resnet152	O	O
pytorch-resnet18	O	O
pytorch-resnet34	O	O
pytorch-resnet50	O	O
pytorch-resnext101_32x8d	O	O
pytorch-resnext50_32x4d	O	O
pytorch-shufflenet_v2_x0_5	X	O
pytorch-shufflenet_v2_x1_0	X	O
pytorch-squeezenet1_0	O	O
pytorch-squeezenet1_1	O	O
pytorch-vgg11	O	O
pytorch-vgg11_bn	O	O
pytorch-vgg13	O	O
pytorch-vgg13_bn	O	O
pytorch-vgg16	O	O
pytorch-vgg16_bn	O	O
pytorch-vgg19	O	O
pytorch-vgg19_bn	O	O
pytorch-wide_resnet101_2	O	O
pytorch-wide_resnet50_2	O	O

ONNC v1.2.0 Release

Release Note

ONNC framework

• [New Feature] add methods for manipulating ComputeOperator input/output links
• [New Feature] add methods for erasing Value in Module
• [New Feature] add new method addOnncIrOptimization() for class TargetBackend
• [New Feature] add new method runOnComputeGraph() for class CustomPass<T>
• [New Feature] add several utility libraries
• [New Feature] add 5 ONNC IR optimization passes
  • divide_globalap_into_aps
  • eliminate_identity
  • expand_batch_normalization
  • propagate_const_with_diff_shape
  • replace_gemm_by_conv
  • split_conv_by_channel
• [Bug fix] fix segmentation fault due to unexpected global opt<T> object initialization order
• [Bug fix] fix name collision when using type LiveInterval
• [Bug fix] remove C++11 incompatible codes
• [Bug fix] fix bugs in default ComputeVisitor::visit() implementation
• [Bug fix] fix ONNC runtime bugs for 12 ONNX model zoo models
• [Bug fix] Add detailed error messages for unsupported ONNX operators

NVDLA Backend

• [New Feature] support more operators
• [New Feature] add NVDLA UMD/KMD patch file (in the nvdla directory)
• [New Feature] add single layer test models for supported 16 ONNX operators (in the single_layer_test directory)
• [Bug fix] avoid creating duplicate AddressListEntry
• [Bug fix] fix sharing memory logics for Reshape
• [Bug fix] fix split Conv algorithms to get correct inference result
• [Bug fix] fix incorrect data & weight bank allocation logics for Conv
• [Bug fix] fix incorrect memory source setting for group Conv
• [Bug fix] fix incorrect weight packing logics for Conv
• [Bug fix] fix incorrect data cube size calculation logics
• [Bug fix] remove redundant MemoryListEntry blocks in NVDLA Loadables
• [Bug fix] fix incorrect NVDLA Loadable task submit logics
• [Bug fix] avoid allocating MemoryListEntry for unused tensors
• [Bug fix] fix bugs for the case where the AveragePool attribute count_include_pad is 0
• [Bug fix] fix segmentation fault for non-biased Conv
• [Bug fix] fix LRN lookup table settings

ONNX Operator Support

Please refer to Supported Operators for more details.

• Support Operators
  • Add new
  • AveragePool
  • BatchNormalization
  • Concat
  • Conv
  • Gemm
  • GlobalAveragePool new
  • Identity new
  • LRN
  • MaxPool
  • Mul new
  • Relu
  • Reshape
  • Softmax
  • Sum
  • Transpose (use in ShuffleNet) new
  • Unsqueeze new

ONNX Model Zoo Support Status

ONNC can successfully compile 12 ONNX models listed in the following table from the ONNX Model Zoo, and run inference on NVDLA virtual platform (with nv_full hardwre configuration) correctly.

Model	Support
bvlc_alexnet	O
bvlc_googlenet	O
bvlc_reference_caffenet	O
bvlc_reference_rcnn_ilsvrc13	O
densenet121	O
inception_v1	O
inception_v2	O
resnet50	O
shufflenet	O
squeezenet	O
vgg19	O
zfnet512	O

ONNC 1.0.0 Release

Release Note

New Features

• NVDLA Backend

  • The first open-source compiler backend that supports NVIDIA Deep Learning Accelerator (NVDLA)

  • Initial release of nv_full hardwre configuration support

  • Support status for the models in the ONNX model zoo - ONNC can compile 6 models and run on NVDLA virtual platform successfully. 2 models are not supported by nv_full configuration. The other 4 models need support for more operators.

    Models	Official NVDLA compiler	ONNC compiler
    bvlc_alexnet	O	O
    bvlc_googlenet	X	O
    bvlc_reference_caffenet	O	O
    bvlc_reference_rcnn_ilsvrc13	O	O
    densenet121	X	Need more operator support
    inception_v1	X	O
    inception_v2	X	Need more operator support
    resnet50	O	O
    shufflenet	X	Need more operator support
    squeezenet	X	Need more operator support
    vgg19	X	nv_full can not support this model
    zfnet512	X	nv_full can not support this model

• Framework Support

  • Interpreter Interface - Target backend now can write a customized interpreter.
  • Vanilla Backend - A template for porting a new backend
  • Statistic API

• Tools

  • ONNI
    • Add more verbose level for debugging or benchmarking (level 1 to 4)
    • Add flag --dry-run : Do not do the inference, just print statistics.
    • Add flag --onnx-opt: Enable onnx optimizer.
    • Add flag -fLinearScanAlgo=<string>: Select linear scan algorithm: first-fit, best-fit. (default is first-fit)
    • Add flag --enable-x86-fuse-conv-relu: Enable x86 fuse conv relu.

• Documentation

  • ONNC-Utilities.md - How to build and run onnc

0.9.3

Executive Summary

1. Complete the x86 interpreter runtime.
2. Enable FuseInplaceValue pass on x86 to further optimize memory footprint.

ONNC preview version - 0.9.2

Executive Summary

1. Complete the x86 memory allocation with liveness analysis information, and
   tested on experimental in-house runtime that will be released for
   next version.
2. Integrate TravisCI with github PR system.

ONNC preview version - rc1

Executive Summary:

1. Finish basic infrastructure of ONNC, including IR, Core, ADT, Support,
   Diagnostic and backends.
2. Enable Sophon backend. Now Sophon backend still use old form of compute
   operators.
3. Provides onnc, onnx2tg and some utilities.
4. Rename TG backend as Sophon backend.

ONNC preview version

First open source release: 0.9.0

Executive Summary:

1. Finish basic infrastructure of ONNC, including IR, Core, ADT, Support,
   Diagnostic and backends.
2. Enable Sophon backend. Now Sophon backend still use old form of compute
   operators.
3. Provides onnc, onnx2tg and some utilities.