sync : ggml (new ops, new backend, etc)

[ggerganov]

No description provided.

[ggerganov]

@slaren Did I mess-up the sync, or is this some issue with Windows:

https://github.com/ggerganov/whisper.cpp/actions/runs/7129122907/job/19412511626

[slaren]

This seems to be caused by the code to register backends automatically on startup. The macros used for this depend heavily on the compiler and can be finicky, but it should work on MSVC. It does work on ggml and llama.cpp, so I am not sure what is different here.

[zshannon]

there's a bug somewhere in this commit for metal. running the whisper.swiftui example on macOS, it compiles then crashes when clicking "transcribe" with a console error "validateComputeFunctionArguments:1056: failed assertion `Compute Function(kernel_soft_max_4): missing buffer binding at index 1 for src1[0].'".

same thing works fine with previous commit 3163090.

sorry to not have more info, the metal bits go way over my head and make me so so grateful for these awesome libraries :)

[ggerganov]

@zshannon Should be fixed on master.

By the way, can you explain how you run the whisper.swiftui example?
When I try to run it, I get the following error, and I cannot figure out how to fix it:

Edit: nevermind - something was wrong with my Xcode project. I re-cloned the repo from scratch and it works now

[landtanin]

Hi @ggerganov , on the latest commit 6335933, I'm still seeing crashes from ggml-metal with the error Compute Function(kernel_soft_max_4): missing buffer binding at index 1 for src1[0].

I naively tried the solution from #1607 but to no avail.

Crashes occur when I run it on an iPhone 13 with the ggml-base model.

---

To make it work on the same phone, I either

• switch to the ggml-tiny
• switch to commit 3163090

Notes: I use a simple wav file with only a few seconds of spoken words. My app code is based on whisper.swiftui

P.S. this can never be said enough, thanks for this amazing project 🙌

[landtanin]

This is fixed in v1.5.2 released 👍

[AuroraWright]

This commit seems to break large-v3 CoreML for me.
I reverted 8171e62 7bc4d22 and this one, and using this audio file https://drive.google.com/file/d/1HWS1pExtOR2mYr0q7w_8ZCl69BjlFZad/view?usp=sharing with large-v3 (command line: -t 10 -pp -osrt -l ja output.wav ) it starts repeating "カタカナで裁判官" at 00:09:30.680 and never recovers. I can consistently reproduce it every time I try
Interestingly, the same issue is not occurring on large-v3 without CoreML

[ggerganov]

@AuroraWright Can you confirm that on latest master using CoreML with large-v3 does not work for you? Does it work if you add the -mc 0 flag?

Does anyone else experience issues on latest master with Metal / CoreML?

[AuroraWright]

@ggerganov yeah with latest master too, it seems mc = 0 fixes it
actually my bad, even before this commit with CoreML it avoids that specific repetition but then eventually breaks/starts repeating at 00:48:54 (same audio).
I also noticed with large-v2+CoreML and latest master it just prints garbage from 50:37 to 51:05 (eg 00:50:39,430 --> 00:50:40,730 ーーーーーーーーーーーーーーーーーーーー ), and everything is fine without CoreML.