Implement scalar sum over all rows in ggml_compute_forward_sum_f32

[xaedes]

I was trying the test-grad0 test from ggml and this way I found a bug in ggml_compute_forward_sum_f32.
Only the sum of the last processed row was written to dst:

https://github.com/ggerganov/llama.cpp/blob/8a0f8673ba1cdc6aa6df27a9fbc698431ca70e8d/ggml.c#L3201-L3210

https://github.com/ggerganov/llama.cpp/blob/8a0f8673ba1cdc6aa6df27a9fbc698431ca70e8d/ggml.c#L6782-L6790

With this PR the sum over all rows is accumulated and then written to dst.

What is not so nice about it, is that the row sum is converted from ggml_float to float, losing precision during accumulation.

---

Was computing single sum over all rows like I did meant by this TODO?
https://github.com/ggerganov/llama.cpp/blob/8a0f8673ba1cdc6aa6df27a9fbc698431ca70e8d/ggml.h#L486-L489

[ggerganov]

Nice catch!

The confusion happened here: ggerganov/whisper.cpp#390
I didn't notice how ggml_vec_sum_f32() was being used in ggml_compute_forward_sum_f32() and made the mistake.

The TODO comment is very old and I have forgotten what it refers to, but for sure it is not related to this bug.
I think it might be related to adding an operator that computes the sums for each row and instead of returning a scalar, it returns [ne1, ne2, ne3, 1] tensor with the sums for each row of the input [ne0, ne1, ne2, ne3] tensor. But not 100% sure

[xaedes]

Ah okay, then we leave that TODO comment for later :)