NN_FUNCTION，build tensorflow model as pytorch nn functional style

tf.layers or slim is too sophisticated to build models efficiently. They all call tf.nn in low level.

We provide NNFunciton wrapping tf.nn functions to simplify building neural network like pytorch, just claiming F = NNFunction(is_training, dtype, data_format).

base model

We evaluate most image classification models using horovod + tf.estimator + tf.data, it stays state of art at most cases.

resnet and resNeXt models use preprocessing of RESNET (resnet ppr. as short) as default. (resize min 255,then crop 224 x 224)

(inference GPU: 1080Ti, without any optimization)

name	STA	top1	top5	inference speed(fp32)	inference speed(fp16)	loss	others
inception_v3	78.8	79.248	94.432	3.661	2.688	1.653	299,ls,e300
,224x224		77.158	93.496	2.251		1.818	224,e120,inception ppr.
se_inception_v3		79.938	94.678	3.817	2.6400	1.661	299,ls,e200
,e120		79.472	94.656	3.788	2.708	1.719
resnet_v1_50		76.174	92.988	2.276		1.411
mobilenet_v1_1.0	72.93	73.23	91.264	1.321		1.382	ls,e200
mobilenet_v2_1.4	75.0	74.536	91.966	2.111		1.391	ls,e120
,dw w/o relu6		74.928	92.192	2.049		1.358	ls,e200
mobilenet_v2_1.0		71.506	90.26	1.637		1.518	ls,e120
,dw w/o relu6	71.8	71.6	90.242	1.337		1.491	ls,e200
resnet_v1_101_D	79.78	79.572	94.818	3.483	2.935	1.229	resnet ppr
,320x320		80.804	95.414	8.642		1.147	320,ls,e120,inception ppr.
resNeXt_50_32x4d_D		78.192	94.014	3.173	2.681	1.265
resNeXt_50_64x4d_D		78.348	94.206	3.899		1.383
resNeXt_101_64x4d_D		79.432	94.6	6.396	5.417	1.342
,320x320		80.304	95.24	13.988		1.302	320,ls,e120,fp16, bs32
resnet_v1_50_D		78.342	94.16	2.520	2.117	1.251	e120,resnet ppr.
,inceptio ppr.	78.48	78.444	94.26	2.803		1.246	e120,inception ppr.
,320x320		78.984	94.664	5.399		1.217	e120, inception ppr. fp16
,se		79.192	94.678	2.998		1.199	e200, inception bbox ppr. fp32, ls
shufflenet_v2_2.0	74.9	74.848	92.146	1.772	1.577	1.334	e200, inception bbox ppr, fp32, ls
,epoch120		74.018	91.738		1.567	1.393	inception bbox ppr, fp32, ls