how to add MKL Packed interface

[tensor-tang]

Need discuss about the python interface how to use mkl packed implements.

As this issue is long, I summarize the key points in Chinese as first:

intel对GRU等layer优化（还是使用MKL库，非MKLDNN库）完并通过测试后，python api封装时，cpu模式（同时开启with_mkl=on），python端是否可以默认调用优化过后的c++ layer？

Background

We plan to add MKL Packed implements for RNN layers,(Recurrent Layer, LSTM, and GRU). related #6512 and #6506 .

They are total different with MKLDNN, and they will be compiled only if WITH_MKL=ON,

Then we got a problem about how to use it from python side?

Solutions

There are two options:

• Add one more flag: use_mkl_packed just like use_mkldnn. This would make things flexible but complex. If user want to use both. They should add both use_mkldnn=True and use_mkl_packed=True.

• Change use_mkldnn ==> use_mkl, and make use_mkl cover both. This can make users happy, Just use_mkl=True is enough, do not need care which one to choose. But it's not flexible enough.

Which one we should choose? Any suggestion is appreciated.

@wangkuiyi @luotao1

[luotao1]

As MKL Packed optimization on RNN are totally different with MKLDNN, and is compiled only when WITH_MKL=ON, thus, above two options are not good enough:

• The first one adds use_mkl_packed, which will make users more confused. Users cannot distinguish the difference between use_mkldnn and use_mkl_packed. If we tell them, when configure RNN, they can use use_mkl_packed, when configure CNN, they can use use_mkldnn, they will be crazy.
• The second one changes use_mkldnn ==> use_mkl. However, in cpu mode, the default library is mklml now, thus, the default value of use_mkl in ON? and default library is both mklml and mkldnn?

I have another option: when MKL Packed optimization on RNN, in cpu mode and with_MKL=ON, can we default use these optimized C++ code to replace the original recurrent layer/LSTMLayer/GRULayer?

@hedaoyuan How do you think about it ? Any suggestion is appreciated.

[tensor-tang]

The second one changes use_mkldnn ==> use_mkl. However, in cpu mode, the default library is mklml now, thus, the default value of use_mkl in ON? and default library is both mklml and mkldnn?

About this suggestion. In CPU mode, default compiling is WITH_MKL=ON, so will build both mklml
and mkldnn, But the default value use_mkl=False. Only if user set use_mkl=True, both MKL pakced function and MKLDNN functions will turn on.

I have another option: when MKL Packed optimization on RNN, in cpu mode and with_MKL=ON, can we default use these optimized C++ code to replace the original recurrent layer/LSTMLayer/GRULayer?

I think this is my option one, just default set use_mkl_packed=true in C++ code. Then users do not need to handle too much options.

[luotao1]

default compiling is WITH_MKL=ON, but default value use_mkl=False

This will make both develops and users confused. As original Paddle c++ code uses MKL library as well, use_mkl=False will make user think that Paddle use openblas library in this case.

[tensor-tang]

OK, I get your point.

It seems option one is better, at least we can separate them.

[luotao1]

Discussed with @hedaoyuan , python api (both v2 and v1) can default use MKL packed C++ codes after testing.

[tensor-tang]

OK, Thx. I see.