Vision Pro front camera streaming to YouTube via HaishinKit

[devon3000]

Hi all. If you're familiar with the Vision Pro you probably are aware that access to the front camera is restricted for privacy reasons. However Apple recently enabled new Enterprise API entitlements which allow front camera access for private apps.

I got the Enterprise API approval recently, and wrote a little demo project to make use of the front-facing camera to stream to YouTube using HaishinKit 2.0.1. Works great and it should be a good jumpstart for any remote support applications.

Project is here, MIT license

Demo video

Since the front camera doesn't present as an AVCaptureDevice but rather as a CameraFrameProvider, we can't use HaishinKit's simple attachVideo method to get a video stream for the RTMP streaming. Instead we have to access individual frames and encode them using the Apple VideoToolbox framework. Frame output from the encoder is sent to HaishinKit via the append method. The microphone gets handled a bit weirdly too. Hopefully the time I spent banging my head against this will save someone else some time!