Deepleapy/src/dnn/check.py:
When units is set to [4,5,2]/[10,6,2], and run grandient check, than the model with softmax as its out_activation will not pass gradient check:
Checking softmax ...
Evaluate the norm of the difference between two dW: 4.796879e-01
Evaluate the norm of the difference between two db: 8.982331e-04
Did not passed the gradient check!
I'am puzzled with it long time.