macOS version crashes after a few seconds of sending video frames

[ldenoue]

I am using a custom video capturer that creates RTCVideoFrame(s) from a CGImage converted to a CVPixelBuffer (to initialize the RTCVideoFrame):

func processSampleBuffer(_ pixelBuffer: CVPixelBuffer) {
        let rtcpixelBuffer = RTCCVPixelBuffer(pixelBuffer: pixelBuffer)
        let timeStampSeconds = CACurrentMediaTime()
        let timeStampNs = lroundf(Float(timeStampSeconds * Double(NSEC_PER_SEC)))
        let videoFrame = RTCVideoFrame(buffer: rtcpixelBuffer, rotation: RTCVideoRotation._0, timeStampNs: Int64(timeStampNs))
        if let videoCapturer = videoCapturer {
            videoCapturer.delegate?.capturer(videoCapturer, didCapture: videoFrame)
        }
    }

After a few seconds (might happen when the video resolution changes to 640x360) but not sure, then I get this crash (hard to debug for me).

Any idea?
By the way, a very similar code works great using your iOS demo sample.
Do you have a macOS version of that sample?

Laurent

[ldenoue]

Follow up: if I limit my frame sizes I send to max 540 pixels (in either width or height), then it runs fine.
Have others seen issues? Do you know if we can force h264 to be used by the encoder to make sure it's using the GPU?

[ldenoue]

More info: specifying "VP8" (or "VP9") as the preferred encoder solves the issue.
Not sure why H264 fails on MacOS.

private static let factory: RTCPeerConnectionFactory = {
        RTCInitializeSSL()
        // from https://stackoverflow.com/questions/66198164/how-to-enable-vp8-on-ios-webrtc
        let videoEncoderFactory = RTCDefaultVideoEncoderFactory()
        for codec in videoEncoderFactory.supportedCodecs() {
           print(codec.name)
           print(codec.parameters)
        }
        videoEncoderFactory.preferredCodec = RTCVideoCodecInfo(name: "VP8")
        //videoEncoderFactory.preferredCodec = RTCVideoCodecInfo(name: "H264", parameters: ["profile-level-id" : "42e01f"])
        let videoDecoderFactory = RTCDefaultVideoDecoderFactory()
        return RTCPeerConnectionFactory(encoderFactory: videoEncoderFactory, decoderFactory: videoDecoderFactory)
    }()

[anthonynevo]

@ldenoue I've been using version M96 on a production app for a few months with H264 with no issues (on iOS however)... until a few weeks ago where the H264 encoder started to fail.
By using VP8, we were able to push a new working version while trying to understand the problem with H264. On our part, it seems to coincide with the release of iOS 15.5. Which version of macOS are you using ? The latest ?
I was hoping that the latest version of webRTC fixes this issue but haven't succeeded (yet) in compiling it for iOS. @stasel is working on having this new version (M102) available, I believe (#36).

[ldenoue]

@anthonynevo I'm on macOS Monterey 12.4 (21F79) and iOS 15.4.1 (on iOS H264 works).

[stasel]

Can you try to reproduce the issue on the latest WebRTC version M103? Maybe it was fixed by the WebRTC folks

[ldenoue]

I updated my package to use the latest 103.0.0 but it still does not work unless I explicitly specify VP8 in videoEncoderFactory.preferredCodec = RTCVideoCodecInfo(name: "VP8")

[mikewoodworth]

Im having exactly the same problem w/ M103... crash in h264 encode on macOS, same app working fine on iOS. The backtrace is very similar

it appears to be a message send to a zombie object. running with zombies enabled seems to confirm:

2022-07-28 00:52:24.429719-0700 video-eo-eo[9790:191435] *** -[Not A Type _cfTypeID]: message sent to deallocated instance 0x1178732c0

[zSoNz]

Still reproduce, even set VP8 doesn't help.

[mikewoodworth]

I have a fix, trying to figure out how to submit to webrtc now, if anyone is interested in building from source I can send a diff

[zSoNz]

I have a fix, trying to figure out how to submit to webrtc now, if anyone is interested in building from source I can send a diff

Yes please 😩

[danilaplee]

videoEncoderFactory.preferredCodec = RTCVideoCodecInfo(name: "VP8")

any updates?

I have a fix, trying to figure out how to submit to webrtc now, if anyone is interested in building from source I can send a diff

can u send?

[mikewoodworth]

my bug report is here, the fix is submitted, you can fine it linked in the comments. hopefully it will land soon.
https://bugs.chromium.org/p/webrtc/issues/detail?id=14688