Add a feature flag for Metal acceleration support

[tazz4843]

Should hopefully fix errors along the lines of the following on macOS:

 Undefined symbols for architecture x86_64:
            "_MTLCopyAllDevices", referenced from:
                _ggml_metal_init in libwhisper_rs_sys-3c0021a7bf9aa3b1.rlib(ggml-metal.m.o)
            "_MTLCreateSystemDefaultDevice", referenced from:
                _ggml_metal_init in libwhisper_rs_sys-3c0021a7bf9aa3b1.rlib(ggml-metal.m.o)
            "_OBJC_CLASS_$_MTLComputePassDescriptor", referenced from:
                objc-class-ref in libwhisper_rs_sys-3c0021a7bf9aa3b1.rlib(ggml-metal.m.o)
          ld: symbol(s) not found for architecture x86_64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

ggml_metal_init: allocating
ggml_metal_init: found device: AMD Radeon Pro 5500M
ggml_metal_init: found device: Intel(R) UHD Graphics 630
ggml_metal_init: picking default device: AMD Radeon Pro 5500M
ggml_metal_init: loading '(null)'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=258 "The file name is invalid."

cc @cenzovit

[codesoda]

Pretty excited to see this in action

[jungsu0717]

Is the issue resolved?
I'm developing a macOS app based on your project.

Undefined symbols for architecture arm64:
            "_MTLCopyAllDevices", referenced from:
                _ggml_metal_init in libwhisper_rs_sys-0c9543eb5bd16852.rlib(ggml-metal.m.o)
            "_MTLCreateSystemDefaultDevice", referenced from:
                _ggml_metal_init in libwhisper_rs_sys-0c9543eb5bd16852.rlib(ggml-metal.m.o)
            "_OBJC_CLASS_$_MTLComputePassDescriptor", referenced from:
                objc-class-ref in libwhisper_rs_sys-0c9543eb5bd16852.rlib(ggml-metal.m.o)
            "_OBJC_CLASS_$_NSBundle", referenced from:
                objc-class-ref in libwhisper_rs_sys-0c9543eb5bd16852.rlib(ggml-metal.m.o)
            "_OBJC_CLASS_$_NSString", referenced from:
                objc-class-ref in libwhisper_rs_sys-0c9543eb5bd16852.rlib(ggml-metal.m.o)
          ld: symbol(s) not found for architecture arm64
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

How can I fix?

[tazz4843]

No one's tested this besides me, with an Apple unsupported macOS system, so I'm hesitant to merge yet.

[jungti1234]

No one's tested this besides me, with an Apple unsupported macOS system, so I'm hesitant to merge yet.

I tested it on my M2 macbook pro, build succeed but there was no change in GPU usage when I use whisper, so I think can't sure it works or not... 😢 I'm will keep try to find a way for success of metal acceleration support.

[garyhai]

I test it on M1 Mac mini, Intel iMac and Intel Macbook. All build succeed and examples work fine. It seems GPU is being utilized as well on M1 Mac.

[tazz4843]

Given that I didn't change anything fundamentally with those merge conflicts, I'm going to go ahead and merge. If it breaks again, let me know.

[jungsu0717]

I just tried it and the build works fine, but there is no log that metal was used.

�What am I missing?

whisper_init_from_file_no_state: loading model from '/ggml-small.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 768
whisper_model_load: n_audio_head  = 12
whisper_model_load: n_audio_layer = 12
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 768
whisper_model_load: n_text_head   = 12
whisper_model_load: n_text_layer  = 12
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 3
whisper_model_load: adding 1608 extra tokens
whisper_model_load: model ctx     =  464.68 MB
whisper_model_load: model size    =  464.44 MB
whisper_init_state: kv self size  =   15.75 MB
whisper_init_state: kv cross size =   52.73 MB
whisper_init_state: compute buffer (conv)   =   19.96 MB
whisper_init_state: compute buffer (encode) =  122.04 MB
whisper_init_state: compute buffer (cross)  =    5.86 MB
whisper_init_state: compute buffer (decode) =   36.17 MB
whisper_full_with_state: auto-detected language: ko (p = 0.991928)

[tazz4843]

That I have no clue about. It could be anything. As far as I can tell, we are linking with the required libraries now, so it should work fine. Like I said, I'm on an Apple-unsupported setup (macOS 14 on a 2012 MBP with legacy Metal support) so I doubt most of what I try will apply to others.

[codesoda]

@jungsu0717 did you specify to use the "metal" feature, ie

[dependencies]
whisper-rs = { git = "https://github.com/tazz4843/whisper-rs", branch = "master", features = ["metal"] }