how to add kernel hint

[jacquesqiao]

problem

user may have some personal preference to choose a certain kind of kernel for an operator, such as force_cpu to use a cpu kernel , use_cudnn to choose a cudnn kernel, use_mkldnn to choose an mkldnn kernel, we need to provide a way to enable the user to do this.

In the current design, we use KernelType

struct KernelType {
  Place place_;
  DataType data_type_;
  LayoutType layout_;
};

to determine the kernel, place_ data_type_ and layout_ are all come from the input of the operator. the user can not config it. GetActualKernelType is used to get this Kernel type, but the design also provides a virtual method GetExpectedKernelType, that user can overload and choose the KernelType they want to use.

so, we should send the information user defined in Python to GetExpectedKernelType to let the user use it.

The problem is, how should we define this information and how should we store it for GetExpectedKernelType to use?

solution

1, Do nothing, let the user add the information they want to operator‘s attribute and get them insideGetExpectedKernelType, this seems can work right. There is a little problem that users may define many kinds of hint for the same purpose, such as force_cpu, use_cpu, cpu for cpu kernel, and use_cudnn, force_cudnn, cudnn_kernel for use of cudnn kernel.

2, pre-define all the needed option and use a single attr key such as kernel_hint for user, this may not be so flexible if the user wants to define some more kind of hint.

I think at least we can predefine some options, such as force_cpu, use_cudnn, use_mkldnn for a user to choose.

[luotao1]

What's the difference between the meaning of use_* and force_*?

[jacquesqiao]

fixed by https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/kernel_hint_design.md