FrostNet: Towards Quantization-Aware Network Architecture Search

Taehoon Kim^1,2, YoungJoon Yoo¹, Jihoon Yang² | Paper | Pretrained Models

1. Clova AI Research, NAVER Corp.
2. Sogang University Machine Learning Lab.

Abstract

INT8 quantization has become one of the standard techniques for deploying convolutional neural networks (CNNs) on edge devices to reduce the memory and computational resource usages. By analyzing quantized performances of existing mobile-target network architectures, we can raise an issue regarding the importance of network architecture for optimal INT8 quantization. In this paper, we present a new network architecture search (NAS) procedure to find a network that guarantees both full-precision (FLOAT32) and quantized (INT8) performances. We first propose critical but straightforward optimization method which enables quantization-aware training (QAT) : floating-point statistic assisting (StatAssist) and stochastic gradient boosting (GradBoost). By integrating the gradient-based NAS with StatAssist and GradBoost, we discovered a quantization-efficient network building block, Frost bottleneck. Furthermore, we used Frost bottleneck as the building block for hardware-aware NAS to obtain quantization-efficient networks, FrostNets, which show improved quantization performances compared to other mobile-target networks while maintaining competitive FLOAT32 performance. Our FrostNets achieve higher recognition accuracy than existing CNNs with comparable latency when quantized, due to higher latency reduction rate (average 65%).

Model performances

ImageNet classification results

• Accuracy comparison with other state of the art lightweight models:

  
  

COCO detection results

• mAP scores comparison on MS COCO val split 2017 with RetinaNet and Faster-RCNN:

  

Pretrained models

• We provide FrostNets' pretrained weights on ImageNet dataset. Note that all the models are trained and evaluated with 224x224 image size. Google Drive

Getting Started

Training your own FrostNet

We trained FrostNets with one of the popular imagenet classification code, rwightman's pytorch-image-models for more efficient training. After including FrostNet's model file into the training code, one can train FrostNets with the command line in training_confs.

Post quantization examples

We also provide post-quantization supported version of rwightman's pytorch-image-models in quanitzation-pytorch-image-models for easier post-quantization with PyTorch.

Training object detection models with FrostNet backbones

We trained FrostNets with one of the popular object detection project, mmdetection for more efficient training. Include frostnet_features.py to mmdetection codes to train models.

StatAssist & Gradboost examples

Supports

• Classification (AlexNet, VGG, Resnet, ShuffleNetV2, Mobilenet V2 & V3) (details)
• Object Detection (TDSOD, SSDLITE-MobileNet V2) (details)
• Semantic Segmentation (ESPNet V1 & V2, Mobilenet V2 & V3) (details)
• Style Transfer (Pix2Pix, CycleGAN) (details)

Implementations

• Our StatAssist implementations can be found in:

  • Classification: line 149 - 164 in here.
  • Object Detection: line 185 - 239 in here.
  • Semantic Segmentation: line 205 - 221 in here.
  • Style Transfer: line 42 - 64 in here.

• Our GradBoost version of optimizers can be found here.

Update

• November 27th, 2020
  • FrostNet, quantization-aware neural network architecture, updated. (details)
• July 29th, 2020
  • Quantized CPU latency results updated. (details)

License

This project is distributed under MIT license.

Copyright (c) 2020-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

How to cite

@misc{kim2020frostnet,
      title={FrostNet: Towards Quantization-Aware Network Architecture Search}, 
      author={Taehoon Kim and YoungJoon Yoo and Jihoon Yang},
      year={2020},
      eprint={2006.09679},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}