[ARM] Suport 32-bits CPUs within ACL integration

[dmitry-gorokhov]

Summary

Current ACL integration is limited with 64bit systems only. The proposal is to extend support on 32bit systems.

Problem statement

OpenVINO users still shows some noticable interest for 32bit ARM systems support with reasonable level of performance. OpenVINO itself can be compiled targetinf 32bit ARM HW, but lack optimized kernels. Same about oneDNN - existing c++ kernels are not compatitive from performance perspective while JIT approach is limited for 64 bit HW. At the same time ACL contains optimized implementations that can be launched on 32bit ARM and as a result demonstrate competitive e2e performance.

Preferred solution

OpenVINO implemented 2 patches inside own oneDNN fork:

1. Enabled oneDNN build on 32Bit ARM systems
2. Extracted ACL primitives into separate backend (while JIT aarch64 kernels are untouched) and started to build this backend on ARM32 systems.

The goal of this ticket is to discuss if there is an interest to extend oneDNN coverage for 32bit ARM systems and understand if the descibed approach can be adopted as a solution in the mainstream.

[theComputeKid]

We can support this effort, if you have patches ready, though we might not have resources to commit to developing it ourselves right now. There are things to think about regarding the infrastructure needed for this, but we can discuss that as part of the review process. Overall, seems interesting.