coreml : revisit instructions and model generation scripts

[ggerganov]

The CoreML creation process has been flaky from the very beginning (#566), mainly because the python packages were not compatible with each other. Things likely have changed since the initial support was added and hopefully are more stable now.

• Update CoreML model conversion/creation instructions
• Add CI workflows to exercise CoreML conversion (can utilize ggml-ci for Apple Silicon nodes)

[furqan4545]

how to integrate .mlmodelc into my swiftui app? I am stuck here? I want to use coreml support. is there any example that I can follow?

[danbev]

The swiftui example currently does not have support for coreml, but the objective-c example does, and it contains a link to how to convert models. Perhaps these could be helpful to implement in swift.

[furqan4545]

The swiftui example currently does not have support for coreml, but the objective-c example does, and it contains a link to how to convert models. Perhaps these could be helpful to implement in swift.

I am very beginner in obj C... Would it be possible for you guys to make swift ui example? I think a lot of people would appreciate that because as saw in many other repos people were requesting the same thing.

[danbev]

Would it be possible for you guys to make swift ui example?

Perhaps we can update the existing whisper.swiftui example similar to whisper.objc. I've created #2907 to track this.

[ggerganov]

Would it be possible for you guys to make swift ui example?

Perhaps we can update the existing whisper.swiftui example similar to whisper.objc. I've created #2907 to track this.

If I am not mistaken, it might be as simple as adding WHISPER_COREML=1 somewhere in the build settings and providing a CoreML model to the project.

[danbev]

If I am not mistaken, it might be as simple as adding WHISPER_COREML=1 somewhere in the build settings and providing a CoreML model to the project.

Nice, I'll give this a try. Thanks

[danbev]

Hmm, this was not as simple because of the introduction of the xcframework. We are no longer compiling in XCode but instead using the compiled library from the xcframework. And Core ML support requires the above mentioned macro to be set when compiling. My initial throught was that we could provide builds for Core ML for macos and ios and include them in the xcframework, but that does not work as the selection is based on platform and architecture, so trying to create the xcframework results in:

A library with the identifier 'macos-arm64_x86_64' already exists.

I'll need to investigate this a little more to understand what our options are.

[ggerganov]

I think we can enable both WHISPER_COREML=ON and WHISPER_COREML_ALLOW_FALLBACK=ON in the CMake builds and provide only this option in the XCFramework. This way, if at runtime it fails to find a CoreML model, it will fallback to not using it.

[danbev]

I think we can enable both WHISPER_COREML=ON and WHISPER_COREML_ALLOW_FALLBACK=ON in the CMake builds and provide only this option in the XCFramework. This way, if at runtime it fails to find a CoreML model, it will fallback to not using it.

That would be much things much easier. Great, I'll try this out.

[danbev]

@furqan4545 We've added an example of using Core ML to the whisper.swiftui example now.

[furqan4545]

@danbev and @ggerganov YOU GUYS ARE HERO. I am extremely grateful.
quick question, is it same branch or different because I can't see any latest commit. Or it's just like I have to change nothing in swift-ui code, but model ?

[danbev]

is it same branch or different because I can't see any latest commit. Or it's just like I have to change nothing in swift-ui code, but model ?

You should be able to use the master branch now. You are correct, there have been no changes to the swift-ui code, the changes have been to the xcframework that now builds with Core ML support. This will enable a Core ML model to be used by the example.

Just make sure that you build/rebuild the xcframework using build-xcframework.se and this was updated the other day to support Core ML. After that you'll need to generate the Core ML model following the instructions in the readme, and then add the model to the whisper.swiftui.demo/Resources/Models and finally start the demo. You should see in the log that Core ML model is being used.

[furqan4545]

@danbev Thank you so much Dan again for the explanation. I will give it a try tomorrow.

[danbev]

One thing I noticed while going through the Core ML support is that currently it is required that the original ggml model is also available on the file system. The ggml model will be loaded first, before the Core ML encoder model. I initially thought that this might be an an oversight, that using a Core ML model would only require the converted model, but I believe the original model is still required for the decoder. Perhaps this could be clarified in the docs.

[ggerganov]

the original model is still required for the decoder. Perhaps this could be clarified in the docs.

Yes, this is correct. We should update the docs to highlight this.

[danbev]

I think this could be considered completed by the following commits:

• Update CoreML model conversion/creationinstructions : 11688b2 ✔
• Add CI workflows to exercise CoreML conversion : 04b9508 ✔