fix plugin media stream renderer save leak #609

hthetiot · 2020-11-23T13:19:39Z

No description provided.

…c to null

…g iosrtcPlugin#MediaStreamRenderer_save() to avoid polutiyion on animation

hthetiot · 2020-11-23T13:22:32Z

2020-11-23 14:20:58.731752+0100 Cordova iOSRTC Sample[2602:1952402] Received memory pressure event 2 vm pressure 0

Received memory pressure event 2 vm pressure 0
2020-11-23 14:21:01.618692+0100 Cordova iOSRTC Sample[2602:1952402] Received memory pressure event 4 vm pressure 1

Received memory pressure event 4 vm pressure 1
2020-11-23 14:21:07.363061+0100 Cordova iOSRTC Sample[2602:1952402] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=2 "Specified target process does not exist" UserInfo={NSLocalizedFailureReason=Specified target process does not exist}>
2020-11-23 14:21:07.363128+0100 Cordova iOSRTC Sample[2602:1952402] [ProcessSuspension] 0x1082ce080 - ProcessAssertion: Failed to acquire RBS Background assertion 'NetworkProcess Background Assertion' for process with PID 2705, error: Error Domain=RBSAssertionErrorDomain Code=2 "Specified target process does not exist" UserInfo={NSLocalizedFailureReason=Specified target process does not exist}
2020-11-23 14:21:07.371271+0100 Cordova iOSRTC Sample[2602:1952402] [ProcessSuspension] 0x1082ce0c0 - ProcessAssertion: Failed to acquire RBS Background assertion 'ConnectionTerminationWatchdog' for process because PID is invalid
2020-11-23 14:21:07.372382+0100 Cordova iOSRTC Sample[2602:1952402] [Process] 0x119025618 - [pageProxyID=13, webPageID=14, PID=2706] WebPageProxy::processDidTerminate: (pid 2706), reason 3
2020-11-23 14:21:07.413052+0100 Cordova iOSRTC Sample[2602:1952402] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}>
2020-11-23 14:21:07.413098+0100 Cordova iOSRTC Sample[2602:1952402] [ProcessSuspension] 0x1082ce140 - ProcessAssertion: Failed to acquire RBS Background assertion 'WebProcess Background Assertion' for process with PID 2706, error: Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}
2020-11-23 14:21:07.425785+0100 Cordova iOSRTC Sample[2602:1952402] [Loading] 0x119025618 - [pageProxyID=13, webPageID=14, PID=2706] WebPageProxy::dispatchProcessDidTerminate: reason = 3

hthetiot · 2020-11-23T13:35:32Z

=================================================================
Main Thread Checker: UI API called on a background thread: -[UIView bounds]
PID: 3066, TID: 1962846, Thread name: (none), Queue name: com.apple.root.default-qos, QoS: 0
Backtrace:
4   Cordova iOSRTC Sample               0x00000001006f81bc $s21Cordova_iOSRTC_Sample25PluginMediaStreamRendererC4save8callback7errbackyySSXE_ySSXEtF + 84
5   Cordova iOSRTC Sample               0x00000001006cf270 $s21Cordova_iOSRTC_Sample12iosrtcPluginC24MediaStreamRenderer_saveyySo20CDVInvokedUrlCommandCFyycfU_ + 100
6   Cordova iOSRTC Sample               0x00000001006c49dc $sIeg_IeyB_TR + 20
7   libdispatch.dylib                   0x0000000100a23b68 _dispatch_call_block_and_release + 32
8   libdispatch.dylib                   0x0000000100a255f0 _dispatch_client_callout + 20
9   libdispatch.dylib                   0x0000000100a27f68 _dispatch_queue_override_invoke + 952
10  libdispatch.dylib                   0x0000000100a382ec _dispatch_root_queue_drain + 364
11  libdispatch.dylib                   0x0000000100a38c94 _dispatch_worker_thread2 + 140
12  libsystem_pthread.dylib             0x00000001cc0ec8cc _pthread_wqthread + 216
13  libsystem_pthread.dylib             0x00000001cc0f377c start_wqthread + 8
=================================================================
Main Thread Checker: UI API called on a background thread: -[UIView isOpaque]
PID: 3066, TID: 1963089, Thread name: (none), Queue name: com.apple.root.default-qos, QoS: 0
Backtrace:
4   Cordova iOSRTC Sample               0x00000001006f81e0 $s21Cordova_iOSRTC_Sample25PluginMediaStreamRendererC4save8callback7errbackyySSXE_ySSXEtF + 120
5   Cordova iOSRTC Sample               0x00000001006cf270 $s21Cordova_iOSRTC_Sample12iosrtcPluginC24MediaStreamRenderer_saveyySo20CDVInvokedUrlCommandCFyycfU_ + 100
6   Cordova iOSRTC Sample               0x00000001006c49dc $sIeg_IeyB_TR + 20
7   libdispatch.dylib                   0x0000000100a23b68 _dispatch_call_block_and_release + 32
8   libdisp2020-11-23 14:34:33.596909+0100 Cordova iOSRTC Sample[3066:1962846] [reports] Main Thread Checker: UI API called on a background thread: -[UIView bounds]
PID: 3066, TID: 1962846, Thread name: (none), Queue name: com.apple.root.default-qos, QoS: 0
Backtrace:
4   Cordova iOSRTC Sample               0x00000001006f81bc $s21Cordova_iOSRTC_Sample25PluginMediaStreamRendererC4save8callback7errbackyySSXE_ySSXEtF + 84
5   Cordova iOSRTC Sample               0x00000001006cf270 $s21Cordova_iOSRTC_Sample12iosrtcPluginC24MediaStreamRenderer_saveyySo20CDVInvokedUrlCommandCFyycfU_ + 100
6   Cordova iOSRTC Sample               0x00000001006c49dc $sIeg_IeyB_TR + 20
7   libdispatch.dylib                   0x0000000100a23b68 _dispatch_call_block_and_release + 32
8   libdispatch.dylib                   0x0000000100a255f0 _dispatch_client_callout + 20
9   libdispatch.dylib                   0x0000000100a27f68 _dispatch_queue_override_invoke + 952
10  libdispatch.dylib                   0x0000000100a382ec _dispatch_root_queue_drain + 364
11  libdispatch.dylib                   0x0000000100a38c94 _dispatch_worker_thread2 + 140
12  libsystem_pthread.dylib             0x00000001cc0ec8cc _pthread_wqthread + 216
13  libsystem_pthread.dylib             0x00000001cc0f377c start_wqthread + 8
atch.dylib                   0x0000000100a255f0 _dispatch_client_callout + 20
9   libdispatch.dylib                   0x0000000100a27f68 _dispatch_queue_override_invoke + 952
10  libdispatch.dylib                   0x0000000100a382ec _dispatch_root_queue_drain + 364
11  libdispatch.dylib                   0x0000000100a38c94 _dispatch_worker_thread2 + 140
12  libsystem_pthread.dylib             0x00000001cc0ec8cc _pthread_wqthread + 216
13  libsystem_pthread.dylib             0x00000001cc0f377c start_wqthread + 8
2020-11-23 14:34:33.597106+0100 Cordova iOSRTC Sample[3066:1963089] [reports] Main Thread Checker: UI API called on a background thread: -[UIView isOpaque]
PID: 3066, TID: 1963089, Thread name: (none), Queue name: com.apple.root.default-qos, QoS: 0
Backtrace:
4   Cordova iOSRTC Sample               0x00000001006f81e0 $s21Cordova_iOSRTC_Sample25PluginMediaStreamRendererC4save8callback7errbackyySSXE_ySSXEtF + 120
5   Cordova iOSRTC Sample               0x00000001006cf270 $s21Cordova_iOSRTC_Sample12iosrtcPluginC24MediaStreamRenderer_saveyySo20CDVInvokedUrlCommandCFyycfU_ + 100
6   Cordova iOSRTC Sample               0x00000001006c49dc $sIeg_IeyB_TR + 20
7   libdispatch.dylib                   0x0000000100a23b68 _dispatch_call_block_and_release + 32
8   libdispatch.dylib                   0x0000000100a255f0 _dispatch_client_callout + 20
9   libdispatch.dylib                   0x0000000100a27f68 _dispatch_queue_override_invoke + 952
10  libdispatch.dylib                   0x0000000100a382ec _dispatch_root_queue_drain + 364
11  libdispatch.dylib                   0x0000000100a38c94 _dispatch_worker_thread2 + 140
12  libsystem_pthread.dylib             0x00000001cc0ec8cc _pthread_wqthread + 216
13  libsystem_pthread.dylib             0x00000001cc0f377c start_wqthread + 8
=================================================================
Main Thread Checker: UI API called on a background thread: -[UIView drawViewHierarchyInRect:afterScreenUpdates:]
PID: 3066, TID: 1962846, Thread name: (none), Queue name: com.apple.root.default-qos, QoS: 0
Backtrace:
4   Cordova iOSRTC Sample               0x00000001006f8254 $s21Cordova_iOSRTC_Sample25PluginMediaStreamRendererC4save8callback7errbackyySSXE_ySSXEtF + 236
5   Cordova iOSRTC Sample               0x00000001006cf270 $s21Cordova_iOSRTC_Sample12iosrtcPluginC24MediaStreamRenderer_saveyySo20CDVInvokedUrlCommandCFyycfU_ + 100
6   Cordova iOSRTC Sample               0x00000001006c49dc $sIeg_IeyB_TR + 20
7   libdispatch.dylib                   0x0000000100a23b68 _dispatch_call_block_and_release + 32
8   libdispatch.dylib                   0x0000000100a255f0 _dispatch_client_callout + 20
9   libdispatch.dylib                   0x0000000100a27f68 _dispatch_queue_override_invoke + 952
10  libdispatch.dylib                   0x0000000100a382ec _dispatch_root_queue_drain + 364
11  libdispatch.dylib                   0x0000000100a38c94 _dispatch_worker_thread2 + 140
12  libsystem_pthread.dylib             0x00000001cc0ec8cc _pthread_wqthread + 216
13  libsystem_pthread.dylib             0x00000001cc0f377c start_wqthread + 8
2020-11-23 14:34:42.174529+0100 Cordova iOSRTC Sample[3066:1962846] [reports] Main Thread Checker: UI API called on a background thread: -[UIView drawViewHierarchyInRect:afterScreenUpdates:]
PID: 3066, TID: 1962846, Thread name: (none), Queue name: com.apple.root.default-qos, QoS: 0
Backtrace:
4   Cordova iOSRTC Sample               0x00000001006f8254 $s21Cordova_iOSRTC_Sample25PluginMediaStreamRendererC4save8callback7errbackyySSXE_ySSXEtF + 236
5   Cordova iOSRTC Sample               0x00000001006cf270 $s21Cordova_iOSRTC_Sample12iosrtcPluginC24MediaStreamRenderer_saveyySo20CDVInvokedUrlCommandCFyycfU_ + 100
6   Cordova iOSRTC Sample               0x00000001006c49dc $sIeg_IeyB_TR + 20
7   libdispatch.dylib                   0x0000000100a23b68 _dispatch_call_block_and_release + 32
8   libdispatch.dylib                   0x0000000100a255f0 _dispatch_client_callout + 20
9   libdispatch.dylib                   0x0000000100a27f68 _dispatch_queue_override_invoke + 952
10  libdispatch.dylib                   0x0000000100a382ec _dispatch_root_queue_drain + 364
11  libdispatch.dylib                   0x0000000100a38c94 _dispatch_worker_thread2 + 140
12  libsystem_pthread.dylib             0x00000001cc0ec8cc _pthread_wqthread + 216
13  libsystem_pthread.dylib             0x00000001cc0f377c start_wqthread + 8

…Renderer.save

hthetiot · 2020-11-23T14:06:03Z

This may only crash if you open Safari Debugger but runned this test for 7 minutes with 2 Local RTCPeerConenction with averary of 190 Mo RAM

var canvasEl;
function TestMediaRenderCapture(videoEl) {
    if (!canvasEl) {
        canvasEl = document.createElement('canvas');
        canvasEl.style.position = 'absolute';
        canvasEl.style.left = 0;
        canvasEl.style.bottom = 0;
        appContainer.appendChild(canvasEl);
    }

    const canvasContext = canvasEl.getContext('2d');

    /*
    var image = new Image();
    image.addEventListener('load', function () {
        canvasEl.width = image.width;
        canvasEl.height = image.height;
        ctx.drawImage(image, 0, 0);
    });
    localVideoEl.render.save(function (data) {
        image.src = 'data:image/jpg;base64,' + data;
    });
    */

    // Resize canvas if videoEl size change 
    if (
        canvasEl.width !== videoEl.videoWidth ||
        canvasEl.height !== videoEl.videoHeight
    ) {
        canvasEl.width = videoEl.videoWidth;
        canvasEl.height = videoEl.videoHeight;
    }
    canvasContext.drawImage(videoEl, 0, 0);
}

var testMediaRenderCatpureAnimateFrame;
function TestMediaRenderCaptureAnimate() {
    TestMediaRenderCapture(localVideoEl);
    testMediaRenderCatpureAnimateFrame = requestAnimationFrame(TestMediaRenderCaptureAnimate);
    //testMediaRenderCatpureAnimateFrame =  setTimeout(TestMediaRenderCapture, parseInt((1000/60)*25));
}

It did crash:


is called. -[FontServicesDaemonManager connection]_block_invoke
2020-11-23 15:03:46.483569+0100 Cordova iOSRTC Sample[3561:1974404] Received memory pressure event 2 vm pressure 0

Received memory pressure event 2 vm pressure 0
2020-11-23 15:04:06.104288+0100 Cordova iOSRTC Sample[3561:1974404] Received memory pressure event 4 vm pressure 1

Received memory pressure event 4 vm pressure 1
2020-11-23 15:04:38.413619+0100 Cordova iOSRTC Sample[3561:1974404] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=2 "Specified target process does not exist" UserInfo={NSLocalizedFailureReason=Specified target process does not exist}>
2020-11-23 15:04:38.413701+0100 Cordova iOSRTC Sample[3561:1974404] [ProcessSuspension] 0x10a5cf5c0 - ProcessAssertion: Failed to acquire RBS Background assertion 'NetworkProcess Background Assertion' for process with PID 3565, error: Error Domain=RBSAssertionErrorDomain Code=2 "Specified target process does not exist" UserInfo={NSLocalizedFailureReason=Specified target process does not exist}
2020-11-23 15:04:38.562165+0100 Cordova iOSRTC Sample[3561:1974404] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}>
2020-11-23 15:04:38.562204+0100 Cordova iOSRTC Sample[3561:1974404] [ProcessSuspension] 0x10a5cf6c0 - ProcessAssertion: Failed to acquire RBS Background assertion 'NetworkProcess Background Assertion' for process with PID 3750, error: Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}
2020-11-23 15:04:38.564204+0100 Cordova iOSRTC Sample[3561:1974404] [ProcessSuspension] 0x10a5cf700 - ProcessAssertion: Failed to acquire RBS Background assertion 'ConnectionTerminationWatchdog' for process because PID is invalid
2020-11-23 15:04:38.565341+0100 Cordova iOSRTC Sample[3561:1974404] [Process] 0x10480ba18 - [pageProxyID=13, webPageID=14, PID=3564] WebPageProxy::processDidTerminate: (pid 3564), reason 3
2020-11-23 15:04:38.574005+0100 Cordova iOSRTC Sample[3561:1974404] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}>
2020-11-23 15:04:38.574055+0100 Cordova iOSRTC Sample[3561:1974404] [ProcessSuspension] 0x10a5cf740 - ProcessAssertion: Failed to acquire RBS Background assertion 'WebProcess Background Assertion' for process with PID 3564, error: Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}

…c { instead of self.commandDelegate.run(inBackground: {

samgabriel · 2020-12-22T16:32:37Z

@hthetiot can you please tag the project so that 6.0.17 will be released into npm
thanks

hthetiot · 2020-12-22T19:20:26Z

@hthetiot can you please tag the project so that 6.0.17 will be released into npm
thanks

I need to release before tag, will do soon.

hthetiot added 3 commits November 23, 2020 14:18

force early GC on Image loaded for Video to Canvas shim by retting sr…

4db305e

…c to null

use pluginMediaStreamRenderer.save async via callback and remove NsLo…

15d1f75

…g iosrtcPlugin#MediaStreamRenderer_save() to avoid polutiyion on animation

build www/cordova-plugin-iosrtc.js

1b5f9cf

hthetiot added this to the 6.0.16 milestone Nov 23, 2020

hthetiot self-assigned this Nov 23, 2020

hthetiot added bug enhancement PluginMediaStreamRenderer labels Nov 23, 2020

use self.commandDelegate.run(inBackground for async pluginMediaStream…

3e0fda1

…Renderer.save

iosrtcPlugin#MediaStreamRenderer_save use DispatchQueue.global().asyn…

f226836

…c { instead of self.commandDelegate.run(inBackground: {

hthetiot modified the milestones: 6.0.16, 6.1.1 Nov 25, 2020

hthetiot merged commit 59866a6 into master Nov 27, 2020

This was referenced Nov 27, 2020

canvas drawImage(videoElem...) cause memory leak via RequestFrameAnimation #582

Closed

use background thread for MediaStreamRenderer_save #594

Closed

hthetiot modified the milestones: 6.1.1, 6.0.17 Jan 20, 2021

hthetiot deleted the task/PluginMediaStreamRendererSaveLeak branch March 1, 2021 16:44

hthetiot mentioned this pull request Aug 29, 2021

Audio stream not switched from speaker #707

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix plugin media stream renderer save leak #609

fix plugin media stream renderer save leak #609

hthetiot commented Nov 23, 2020

hthetiot commented Nov 23, 2020

hthetiot commented Nov 23, 2020

hthetiot commented Nov 23, 2020

samgabriel commented Dec 22, 2020

hthetiot commented Dec 22, 2020

fix plugin media stream renderer save leak #609

fix plugin media stream renderer save leak #609

Conversation

hthetiot commented Nov 23, 2020

hthetiot commented Nov 23, 2020

hthetiot commented Nov 23, 2020

hthetiot commented Nov 23, 2020

samgabriel commented Dec 22, 2020

hthetiot commented Dec 22, 2020