GPU mode does not speed up, actually, even worse than CPU mode...

[baileyqbb]

Platform: Rockchip RK3399.

I have setup Ubuntu 16.04 environment with other stuff needed for CaffeOnACL, by following the installation instructions.

While trying to test the classification using ./distribute/bin/classification_profiling_gpu.bin, the processing time is much longer than using ./distribute/bin/classification_profiling.bin.
e.g.
googlenet--> cpu: ~0.54s GPU: ~6s
mobilenet--> cpu: ~0.42s GPU:~2.6s
squeezenet--> cpu: ~0.16s GPU:~2.9s

I have updated the mali libraries from https://github.com/rockchip-linux/libmali and all the unit tests passed.

Checked /dev/mali0, it exists.

Any ideas why this happened?

[anwesha94]

Hi, I have also run on both cpu and gpu. GPU performance is worse than CPU always.
Refer to the performance report CaffeOnACL has provided: https://github.com/OAID/CaffeOnACL/blob/master/acl_openailab/performance_report.pdf

[baileyqbb]

@anwesha94 Thanks for your kind reminder! Indeed the GPU mode is slower than CPU mode. Then may I understand in this way that GPU here is unnecessary, since in the mix mode, only OpenBlas and NEON are used to accelerate the processing? Will there be better performance if using a processor with more cores, say 8? (rk3399 has 6 cores in total)

[daeinki]

I also measured the CaffeOnACL performance and compared CPU and GPU performance.
For the performance measurement, I used image classification applications - classification_profiling_gpu.bin and classification_profiling.bin - with AlexNet and SqueezeNet.

As a result, GPU is slower than CPU in all cases about 3 times!!!.
Test env: Galaxy Note4(Exynos5433 with MALI-T760).

There are also two strange results.

1. CaffeOnACL is slower than original Caffe about 2 times!!!
2. with SqueezeNet, performance no change!!!

• with CPU mode SqueezeNet shows about 2 times faster than AlexNet
  but with GPU mode there is no performance change.

[baileyqbb]

I am guessing one of the reasons that the GPU mode is much slower than CPU mode is that the MALI-GPU's ram is not big enough to contain the whole model for the processing. Thus, it needs to copy partial of the model from cpu to gpu and then back to cpu several times for one forward processing, which will cost more time than running on multi-core cpu.

[daeinki]

Thanks for reply. However, MALI GPU has no internal memory so it uses system memory instead. And Exynos5433 SoC has 3GB system memory.