-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CoreML + calls to whisper_full result in increased memory usage (apparent leak) #1202
Comments
Update: Wrapping the call to void whisper_coreml_encode(
const whisper_coreml_context * ctx,
float * mel,
float * out) {
MLMultiArray * inMultiArray = [
[MLMultiArray alloc] initWithDataPointer: mel
shape: @[@1, @80, @3000]
dataType: MLMultiArrayDataTypeFloat32
strides: @[@(240000), @(3000), @1]
deallocator: nil
error: nil
];
@autoreleasepool{
whisper_encoder_implOutput * outCoreML = [(__bridge id) ctx->data predictionFromLogmel_data:inMultiArray error:nil];
memcpy(out, outCoreML.output.dataPointer, outCoreML.output.count * sizeof(float));
}
} |
Interesting - would you be interested in opening a PR with this change? |
Sure, created a PR. Just for reference, the proposed solution was found on this apple developer forum post where a similar issue was encountered. |
Calls to whisper_full when using CoreML appear to be leaking memory (~5.88mb per call on medium model). Transcription results seem to be fine. This behavior was observed calling the
stream
example. It was not observed when CoreML is disabled.Environment
whisper.cpp 21e8c67
Mac Mini, M2, MacOS 13.4
Build/Run
Debug Info
Every call to whisper_full seems to be allocating a new MLMultiArray. It isn't reported as a leak, because apparently there is still someone holding a reference to it. Nevertheless, every successive call allocates a new one, without releasing the previous one.
The screenshot above shows a debug session on
./stream -m ./models/ggml-medium.bin
. These 5.88mb allocations occur when callingwhisper_coreml_encode
and are never freed.I wonder if this could be related to #797 or #910. Tried to find a solution myself but i have zero experience with ObjC. Any insight is appreciated!
The text was updated successfully, but these errors were encountered: