Refactor the convolution-related code

[hedaoyuan]

For some reason, we need to reconstruct the convolution-related code #2177 . In addition to performance considerations, how to make paddle convolution-related code clearer and more readable is the primary purpose.

So, the whole refactoring process needs to figure out the following things:

• Draw Paddle's current convolutional module code structure, as well as ugly and not clear enough code;
• Draw the reconstructed code structure.
• Defines the data structure of the input, output and filter.(NCHW or NHWC? filter?)
• Output size calculation. (At present, there are several places that define the calculation method of the output size.)
• Optimize temporary memory. (卷积计算过程中会需要一些临时内存，比如exconv实现中，需要一个expandInput_来存储im2col后的临时数据；但是，并不需要每个卷积Layer实例都保存这样一块内存，虽然当前每个exconv layer保存一个expandInput_对象的方法，在多线程中，可以使得线程计算，但是同样也消耗的大量内存。)
• A design document on how to reconstruct the convolution part of the code;
• Migrate the code based on sgemm and cudnn to the new convolution module.
• Adding a convolution implementation based on NNPACK;

[qingqing01]

Defines the data structure of the input, output and filter.(NCHW or NHWC? filter?)

NNPACK only supports NCHW. cuDNN supports NCHW by default. Maybe we can only support NCHW layout.

Output size calculation. (At present, there are several places that define the calculation method of the output size.)

It's better to calculate output size each mini-batch.

[hedaoyuan]

Paddle's current convolutional module code structure

• The three modules contain convolution, inherited from class Layer, class Projection, class Operator.
• The solid line indicates the inheritance relationship, and the dotted line indicates the call relation.
• ConvBaseLayer, ConvBaseProjection, and ConvBaseOperator all contain convolution parameters that have been converted from ConvConfig.

[hedaoyuan]

Draw the reconstructed code structure

• The refactoring described here does not involve modifying the current NN-Layer structure. Just move the convolution forward / backward correlation calculation to the Function.
• ConvBaseFunction contains the basic parameters of convolution.
• ConvGemmFunction and ConvCudnnFunction are two different implementations of convolution. And can add more different implementations.

[hedaoyuan]

Merge some ConvLayer

At present ConvLayer and ConvTransLayer are written two implementations. For example, ExpandConvLayer and ExpandConvTransLayer are implemented in ExpandConvLayer.cpp and ExpandConvTransLayer.cpp files respectively. By comparison found that the two code logic is basically the same. After adding the ConvFunction, we can combine the two codes into one code.

At present

// file: ExpandConvLayer.cpp
REGISTER_LAYER(exconv, ExpandConvLayer);
bool ExpandConvLayer::init(const LayerMap &layerMap,
                           const ParameterMap &parameterMap) {
  /* Initialize the basic convolutional parent class */
  ExpandConvBaseLayer::init(layerMap, parameterMap);
  return true;
}
...

// file: ExpandConvTransLayer.cpp
REGISTER_LAYER(exconvt, ExpandConvTransLayer);
bool ExpandConvTransLayer::init(const LayerMap &layerMap,
                                const ParameterMap &parameterMap) {
  /* Initialize the basic convolutional parent class */
  ExpandConvBaseLayer::init(layerMap, parameterMap);

  return true;
}
...

Use ConvFunction

// file: ExpandConvLayer.cpp
REGISTER_LAYER(exconv, ExpandConvLayer);
REGISTER_LAYER(exconvt, ExpandConvLayer);
bool ExpandConvLayer::init(const LayerMap &layerMap,
                           const ParameterMap &parameterMap) {
  /* Initialize the basic convolutional parent class */
  ExpandConvBaseLayer::init(layerMap, parameterMap);
  if (config.type() == "exconv") {
    forwardFunction = expandFwdOnce;
    backwardFunction = bpropActs
  } else { /* config.type() == "exconvt" */
    forwardFunction = bpropActs;
    backwardFunction = expandFwdOnce;
  }
  return true;
}
...