ERROR:TwilioVideo:[Platform]:Fatal runtime capture error

[MBuLoft1]

Context :

• Our app needs to alternate between the use of the Twilio Video and the use of another piece of code that we have that also does video. We never use both at the same time, but need to switch between one implementation and the other during an application session.
• Furthermore, we are using a TVIVideoRenderer with Twilio and we are also using a frame renderer by means of AVCaptureSession addOutput on the other video implementation.
• Using Twilio Video 1.2.1 for iOS.

The Problem :

• We sometimes get the following error when going from the non-Twilio video to the other video implementation, as if something is not released on the Twilio side.

2017-07-13 11:37:51.281565-0300 MyApp[35204:9400844] ERROR:TwilioVideo:[Platform]:Fatal runtime capture error: Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSUnderlyingError=0x174245d30 {Error Domain=NSOSStatusErrorDomain Code=-12785 "(null)"}, NSLocalizedFailureReason=An unknown error occurred (-12785), NSLocalizedDescription=The operation could not be completed}, code -11800

The Workaround :

• The code below shows the workaround that we found so that the problem does not happen. The workaround is at disconnection from the room : (a) disconnect from the room then (b) connect to the room but using a local track that does not have a renderer, finally (c) disconnect from the room.

The Question :

• Is there something else we need to do ?...
• ...or... is there something that is not released when disconnecting from the room the first time (a) and that finally gets released when re-connecting/disconnecting to/from the room without renderer (b and c) ? If so, can it be fixed in a future release so we do not have to use the workaround ?

Thank you,
Martin

Example of connecting to the room with a local track to which as renderer is attached

class MyTwilioCamera : ... {
    ...
    var localVideoTrack = TVILocalVideoTrack.init(capturer: self._camera!)
    ...
    let connectOptions = TVIConnectOptions.init(token: myAccessToken) { (builder) in
        builder.videoTracks = [localVideoTrack]
        builder.roomName = roomName
    self._room = TwilioVideo.connect(with: connectOptions, delegate: self)
    ...
}
extension MyTwilioCamera : TVIVideoRenderer {
    func renderFrame(_ frame: TVIVideoFrame) { }
    func updateVideoSize(_ videoSize: CMVideoDimensions, orientation: TVIVideoOrientation) { }
}

Example of disconnecting from the room. If we do just step (a) and not steps (b,c), we get the said error. If we do (a, b, c), we do not get the error.

// (a) Disconnect from the room that currently has the local track configured with a capturer that has the renderer
self._room!.disconnect()
// NOTE: would get the said error if did not do (b) and (c) below
// (b) Connect to a room that has a default local track with no custom renderer
let connectOptions = TVIConnectOptions.init(token: myAccessToken) { (builder) in
        builder.videoTracks = [TVILocalVideoTrack]() // a default track without a renderer
        builder.roomName = roomName
self._room = TwilioVideo.connect(with: connectOptions, delegate: self)
// (c) Finallly disconnect because this is our intent
self._room!.disconnect()

[paynerc]

Martin,

Thanks for reporting this issue and with such detailed information. I will start digging into this to see if I can reproduce it and see if there is something that we can do internally to alleviate this from happening.

I'll be in touch!

Ryan

[piyushtank]

@MBuLoft1 Based the on the descriptions it seems like the AVCaptureSession created by Video SDK when you connected to a Room (with video track added), is conflicting with the other one in your app.

To solve this problem, can you delete the TVILocalVideoTrackand stop theTVICameraCapturer` before start interacting with the camera using other code in your app?

Here is code snippet to clean up camera after disconnect -

[self.room disconnect];
self.cameraCapturer = nil
self.videoTrack = nil;
self. localVideoTrack = nil

Regards,
Piyush

[MBuLoft1]

Thank you for your answer, but it is something that I had tried before (and I have now retried it) but it generates the same error.

[piyushtank]

@MBuLoft1 Can you make sure that self.localParticipant is deleted as well?

self.cameraCapturer = nil
self.videoTrack = nil
self.localVideoTrack = nil
self.localParticipant = nil // Make sure local participant is deleted
self.room = nil

On Xcode, you can check Memory Graph Hierarchy to confirm if the Twilio Video SDKobjects are cleaned up properly and the AVACpatureSession is not being held after the clean up -

If this does not help, is it possible for you to share the code which could reproduce the problem reported?

Thanks,
Piyush

[chainans]

Hi, I think I encounter the same issue and by adding self.cameraCapturer?.stopCapture() just before make it nil resolve the problem in my case.

[serjooo]

I was having a similar issue while switching out camera handlers as well, since in my app I wanted to show the user a preview of his camera before he started video chatting. Each of these were in two separate two View Controllers. I was using https://github.com/NextLevel/NextLevel to show the preview and then I had a gesture where I would tap the view and go to the actual video chatting controller and initialize the TVIVideoView as a preview view on that controller and this was seamless. Keep in mind that ofcourse I would stop the NextLevels capture session on viewWillDisappear. However, while going back, with attempting to stop the TVICameraCapturer and removing the TVILocalVideoTrack and all that is suggested in this post, the other camera couldn't instantly capture the camera session. Thus, I changed my solution to use just the Twilio preview view initialization code in both view controllers, because really I didn't need to do much more with the camera than just show a preview. If anyone has the same issue as me, I made this github gist that handles this well and organizes all TwilioVideo related code. https://gist.github.com/serjooo/d5d7c4009a4a9327d7dda315bc42c86b

As for the problem at hand, I'm pretty sure that the Twilio camera session is not being properly stopped or instantly and maybe needs some refinement, because I could instantly switch from NextLevel's camera to Twilio's, however, the inverse was not possible.

[piyushtank]

@serjooo I am sorry for the inconvenience. Looking at your code, it seems TwilioVideoManager is a singleton class. Can you make sure TwilioVideoManager has cleaned up the TwilioVideo objects (mentioned in the previous post) before switching to NextLevel preview? You can make sure this by looking at the Memory Graph when NextLevel ViewController's viewWillApear gets called.

[serjooo]

@ptankTwilio I switched the TwilioVideoManager to a singleton after I faced the problem and removed the NextLevel dependency. I switched it to a signleton so that TwilioVideoManager instance always handles the preview view sent to it. Before, I would only create an instance only when I instantiated the second view controller.

[piyushtank]

Apologies for not following up on this.

Closing this old ticket, as we shipped new VideoSource APIs. Please reopen this or open a new ticket if you run into an issue with latest information.