[TOPI] Tunable Template for Conv2D HWCN on CUDA

[comaniac]

This PR has done the following:

• Enable HWCN layout for conv2d in Relay.

Now we can assign HWCN layout when creating a conv2d op using Relay:

out = relay.nn.conv2d(data, weight, ..., data_layout='HWCN', kernel_layout='HWIO')

AutoTVM can also extract the conv2d tasks with HWCN layout. It can be integrated into the layout conversion pass in the future.

• Make the TOPI schedule of conv2d HWCN on CUDA tunable.

The tunable schedule is based on the original one and the original config is now the fallback, so it won't change the performance if users do not use AutoTVM. Here are the performance results on p3 of some conv2d workloads with different batch sizes. The scalability is defined as (Throughput A / Throughput B) / (Batch Size A / Batch Size B), so 1 means prefer scale. Note that if the config space is less than 2,000 then we stopped after exploring the whole space. We can see from the results that the tuned schedule achieves on average 3.5x speedup and 0.98 scalability.

N	C	H	W	O	KH	KW	Stride	Padding	Space	Default (GFLOP/s)	Scalability	After 2000 Trials (GFLOP/s)	Scalability	Speedup
1	3	224	224	32	3	3	(2, 2)	(1, 1)	52	252.57	1.00	366.1	1.00	1.45
4	3	224	224	32	3	3	(2, 2)	(1, 1)	520	879.11	0.87	2113.74	1.66	2.40
8	3	224	224	32	3	3	(2, 2)	(1, 1)	1768	2773.76	1.58	4803.27	0.72	1.73
16	3	224	224	32	3	3	(2, 2)	(1, 1)	2704	4127.37	0.74	6922.33	0.97	1.68
32	3	224	224	32	3	3	(2, 2)	(1, 1)	3796	4129.1	0.50	7222.31	1.04	1.75
1	192	14	14	64	1	1	(1, 1)	(0, 0)	73	130.99	1.00	342.58	1.00	2.62
4	192	14	14	64	1	1	(1, 1)	(0, 0)	730	517.31	0.99	1902.55	1.41	3.68
8	192	14	14	64	1	1	(1, 1)	(0, 0)	2482	1813.46	1.75	5310.76	0.80	2.93
16	192	14	14	64	1	1	(1, 1)	(0, 0)	3796	4942.23	1.36	6574.4	0.45	1.33
32	192	14	14	64	1	1	(1, 1)	(0, 0)	5329	5625.78	0.57	8945.74	1.20	1.59
1	1280	1	1	1000	1	1	(1, 1)	(0, 0)	80	10.48	1.00	88.11	1.00	8.41
4	1280	1	1	1000	1	1	(1, 1)	(0, 0)	800	40.82	0.97	332.95	0.97	8.16
8	1280	1	1	1000	1	1	(1, 1)	(0, 0)	2720	158.78	1.94	898.16	0.69	5.66
16	1280	1	1	1000	1	1	(1, 1)	(0, 0)	4160	397.55	1.25	1719.73	0.76	4.33
32	1280	1	1	1000	1	1	(1, 1)	(0, 0)	5840	570.4	0.72	2707.75	1.10	4.75

[comaniac]

@Laurawly @kevinthesun @icemelon9 please help review this PR. Thanks.

[Laurawly]

Overall LGTM, some minor comments.

[Laurawly]

Remove comment

[Laurawly]

@merrymercy Do we need this warning?

[comaniac]

The reason we want to remove this warning is that AutoTVM may trigger it with some candidate configs in the tuning process and it looks annoying. If we really need to keep this warning, one alternative solution is trying to hide all warnings in AutoTVM, although I am not sure if it is doable since they are all managed by the same logging system.