[onert] Optimize BatchMatMul kernel in cpu backend

[ragmani]

Let's optimize BatchMatMul kernel in cpu backend
Currently BatchMatMul kernel in cpu backend is not optimized yet.

[zetwhite]

I'm interested in this task.
But I couldn't catch the details.
Could you share any background context?

[ragmani]

Could you share any background context?

The background is to support nmt model now. For now, though, I think this task will be useful in the future.

[chunseoklee]

Could you share any background context?

There is a long story about detailed context. In short, this is a part of work to support transformer-based model (in this case, Machine Translation) For reference, please see link

[glistening]

@tomdol continuation from #14305 (comment)

We need optimized batch matmul for arm32.

For transposed batch matmul, I confirmed it works and is enough using ggml_mulmat.

For normal batch matmul like torch.bmm, which has b × n × m and b × m × p tensors for inputs, we need optimized kernel.

You may bring from other open source kernels (xnnpack, kleidiai, ...) or write by yourself. ( I am not sure that these has the optimized kernel what we need.)

We have an alternative option (e.g. insert transpose before normat batch matmul, and use ggml_mulmat). Thus, if you think that this task takes much time or seems not to be better than alternative approach, you don't need to take this task.