Different results in caffe-android-lib and caffe-lib

[Syndrome777]

Hi, All,

I'm so sorry that I got a different result during using caffe-android-lib.

My task is Classification, and I got the result in softmax layer in my android device which is below:
Label 0, Label 1, Label2, Label3
0.00697519, 0.0600692, 0.904507, 0.0284484

But meanwhile, I got the result in softmax layer in my computer by caffe-lib, is below:
Label 0, Label 1, Label2, Label3
0.008724 , 0.04197951, 0.93302566, 0.01627085

They both use the same model and the same test image.

I don't know if the dropout layer is useful during Test period.
So if there is a bug? Does any have the same problem?

[sh1r0]

@Syndrome777 Did u use Eigen to build caffe on your computer? And, dropout is not applied during test phase.

[Syndrome777]

@sh1r0 Thank you for your response. No, I use caffe's default BLAS(OpenBLAS/CUDA). Is Eigen the reason of this different result ? It seems that this difference is unacceptable.

[sh1r0]

Oh, it could be related to the issue mentioned in BVLC/caffe#2619 (comment). I think my wrappers in mkl_alternate.hpp and math_functions.cpp might be buggy somewhere. Sorry for that. If you want the result to be more precise, maybe you can refer to this to get pre-built OpenBLAS, which is much slower than Eigen, as a workaround.

[sh1r0]

@Syndrome777
I tried to build caffe with both OpenBLAS and Eigen, and test on my pc with the script below.

./build/examples/cpp_classification/classification.bin \
models/bvlc_reference_caffenet/deploy.prototxt \
models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel \
data/ilsvrc12/imagenet_mean.binaryproto \
data/ilsvrc12/synset_words.txt \
examples/images/cat.jpg

And the results shows no difference between the two builds.
EDIT: I also tested on mobile and get the same results.

• openblas_pc.log

---------- Prediction for examples/images/cat.jpg ----------
0.3134 - "n02123045 tabby, tabby cat"
0.2380 - "n02123159 tiger cat"
0.1235 - "n02124075 Egyptian cat"
0.1003 - "n02119022 red fox, Vulpes vulpes"
0.0715 - "n02127052 lynx, catamount"

• openblas_mobile.log

---------- Prediction for /sdcard/caffe_mobile/cat.jpg ----------
0.3134 - "n02123045 tabby, tabby cat"
0.2380 - "n02123159 tiger cat"
0.1235 - "n02124075 Egyptian cat"
0.1003 - "n02119022 red fox, Vulpes vulpes"
0.0715 - "n02127052 lynx, catamount"

• eigen_pc.log

---------- Prediction for examples/images/cat.jpg ----------
0.3134 - "n02123045 tabby, tabby cat"
0.2380 - "n02123159 tiger cat"
0.1235 - "n02124075 Egyptian cat"
0.1003 - "n02119022 red fox, Vulpes vulpes"
0.0715 - "n02127052 lynx, catamount"

• eigen_mobile.log

---------- Prediction for /sdcard/caffe_mobile/cat.jpg ----------
0.3134 - "n02123045 tabby, tabby cat"
0.2380 - "n02123159 tiger cat"
0.1235 - "n02124075 Egyptian cat"
0.1003 - "n02119022 red fox, Vulpes vulpes"
0.0715 - "n02127052 lynx, catamount"

How did you test on your desktop?

[sh1r0]

@Syndrome777 Finally, OpenBLAS is fully (?) supported, please refer to the latest master branch (3ac46b0).

[Syndrome777]

Hi, @sh1r0 , Thank you for your help.
Now I get the same result as you.
Thank you so much again.

[StephenLeeUSTC]

@sh1r0 Hello, Could you tell me how do you get this result ? If this result is run in an android phone and if there is any scripts?

---------- Prediction for /sdcard/caffe_mobile/cat.jpg ----------
0.3134 - "n02123045 tabby, tabby cat"
0.2380 - "n02123159 tiger cat"
0.1235 - "n02124075 Egyptian cat"
0.1003 - "n02119022 red fox, Vulpes vulpes"
0.0715 - "n02127052 lynx, catamount"

Thank you !!!