Convolution weights initialization with Gabor Filters

What is Gabor filler?

Gabor filler is new implementation of weights filler, dedicated for first convolution layer weights initialization. Nevertheless it can be used on any other convolution layer as well. It is based on Gabor Filters, which are described at: https://en.wikipedia.org/wiki/Gabor_filter. The formula used to compute Gabor Filter is:

where

All the parameters are generated by internal algorithm based on convolution kernel shape including: number of kernels, its size and convolution stride. Intel® Distribution of Caffe* uses this filler to speedup training process of most of common DNN topologies like: Cifar, Alexnet, GoogLeNet, VGG, Resnet.

##How to use Gabor filler with Intel® Distribution of Caffe*? To enable Gabor filler for first convolution layer it is needed to change type of weight_filler in convolution_param. Example of AlexNet with Gabor filler enabled is presented below:

layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 96
    kernel_size: 11
    stride: 4
    weight_filler {
      type: "gabor"
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}

---

*Other names and brands may be claimed as the property of others