Mxnet 1.3 Bug : Prelu activation compution fault in expand_shape function

[StOnEGiggity]

Description

when input shape of prelu layer meets the condition where the height or the width of input is the same as channel, like shape (1, 64, 64, 128), it will cause broadcast multiply in the third dimension. There may be wrong in https://github.com/apache/incubator-mxnet/blob/9dd5edd8e6ec28a97c6ba9bda51b2b9c7a88971b/src/operator/leaky_relu-inl.h#L287-L289

Environment info (Required)

----------Python Info----------
Version : 3.5.2
Compiler : GCC 5.4.0 20160609
Build : ('default', 'Nov 23 2017 16:37:01')
Arch : ('64bit', 'ELF')
------------Pip Info-----------
Version : 8.1.1
Directory : /usr/lib/python3/dist-packages/pip
----------MXNet Info-----------
Version : 1.3.0
Directory : /mnt/truenas/scratch/tianqi.tang/incubator-mxnet-syncbn/python/mxnet
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform : Linux-4.4.0-116-generic-x86_64-with-Ubuntu-16.04-xenial
system : Linux
node : rnd-coco
release : 4.4.0-116-generic
version : #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018
----------Hardware Info----------
machine : x86_64
processor : x86_64
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 48
On-line CPU(s) list: 0-47
Thread(s) per core: 2
Core(s) per socket: 12
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
Stepping: 2
CPU MHz: 2599.988
CPU max MHz: 3100.0000
CPU min MHz: 1200.0000
BogoMIPS: 4601.31
Virtualization: VT-x
Hypervisor vendor: vertical
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 30720K
NUMA node0 CPU(s): 0-11,24-35
NUMA node1 CPU(s): 12-23,36-47
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single retpoline kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts
----------Network Test----------
Setting timeout: 10
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0157 sec, LOAD: 1.2276 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0702 sec, LOAD: 1.7148 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.0169 sec, LOAD: 0.9205 sec.
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0086 sec, LOAD: 2.3017 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0086 sec, LOAD: 0.7714 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0167 sec, LOAD: 3.6228 sec.

Minimum reproducible example

import mxnet as mx                                                                                                                                                                                                 
print(mx.__version__)                                                                                                                                                                                              
import numpy as np                                                                                                                                                                                                                                                                                                                                                                                                                     
RELU3 = mx.symbol.LeakyReLU                                                                                                                                                                                                                                                                                                                                                                         
gamma=np.random.normal(size=(64,))                                                                                                                                                                                
print('gamma',gamma)                                                                                                                                                                                               
gamma=mx.nd.array(gamma, ctx=mx.gpu(7))                                                                                                                                                                            
def single_dev_consist():                                                                                                                                                                                         
     data = mx.sym.var("data")                                                                                                                                                                                      
     leaky_gamma = mx.sym.var("prelu3_gamma")                                                                                                                                                                                                                                                                                                                                                                                       
     relu3 = RELU3(name='prelu3', act_type='prelu', data=data, gamma=leaky_gamma)                                                                                                                                   
     relu3_exe = relu3.simple_bind(mx.gpu(7), data=(1,64,64,128))                                                                                                                                                                                                                        
     d = mx.nd.array(np.random.normal(size=(1,64,64,128)), ctx=mx.gpu(7))                                                                                                                                                                                                                                                                                                          
     print('data', d[0, :, 0, 0])                                                                                                                                                                                   
     relu3_exe.forward(is_train=False, data=d, prelu3_gamma=gamma)                                                                                                                                                  
     output1 = relu3_exe.outputs[0].asnumpy()                                                                                                                                                                                                                                                                                                                                           
     print('output',output1[0,:,0,0])                                                                                                                                                                               
     print("\n")                                                                                                                                                                                                    

if __name__ == "__main__":                                                                                                                                                                                         
     single_dev_consist()                 

What have you tried to solve it?

1. one naive solution is to specify the 2nd dimension, like
   (s >= 0 && i == 1 && (dst[i] == src[s] || src[s] == 1)

[lanking520]

Hi @StOnEGiggity thanks for your issues and pointing out the operator, we will start looking into this. @haojin2 @eric-haibin-lin

@mxnet-label-bot could you please add [Bug, operator] here?

[apeforest]

Hi @StOnEGiggity if you already identified the root cause and knew the fix, would you mind making the change directly and contribute to this repository? If so, we are more than happy to review your pull request. but also please don't feel obliged :)

[apeforest]

@sandeep-krishnamurthy @anirudh2290 This issue has been fixed by the requester. Please close it. Thanks!