2.13.0
2.13.0 (March 3, 2021)
New Features
Video Processor API Pilot (Chrome only)
-
You can now register a
VideoProcessor
with a VideoTrack in order to process its video frames. In a LocalVideoTrack, video frames are processed before being sent to the encoder. In a RemoteVideoTrack, video frames are processed before being sent to the attached<video>
element(s). TheVideoProcessor
should implement the interface shown below. (VIDEO-3560, VIDEO-3561)abstract class VideoProcessor { abstract processFrame(inputFrame: OffscreenCanvas) : Promise<OffscreenCanvas | null> | OffscreenCanvas | null; }
A VideoTrack provides new methods addProcessor and removeProcessor which can be used to add and remove a VideoProcessor. It also provides a new property
processor
which points to the current VideoProcessor being used by the VideoTrack. For example, you can toggle a blur filter on a LocalVideoTrack as shown below.import { createLocalVideoTrack } from 'twilio-video'; class BlurVideoProcessor { private readonly _outputFrameCtx: CanvasRenderingContext2D; private readonly _outputFrame: OffscreenCanvas; constructor(width: number, height: number, blurRadius: number) { this._outputFrame = new OffscreenCanvas(width, height); this._outputFrameCtx = this._outputFrame.getContext('2d'); this._outputFrameCtx.filter = `blur(${blurRadius}px)`; } processFrame(inputFrame: OffscreenCanvas) { this._outputFrameCtx.drawImage(inputFrame, 0, 0); return this._outputFrame; } } // Local video track createLocalVideoTrack({ width: 1280, height: 720 }).then(track => { const processor = new BlurVideoProcessor(1280, 720, 5); document.getElementById('preview').appendChild(track.attach()); document.getElementById('toggle-blur').onclick = () => track.processor ? track.removeProcessor(processor) : track.addProcessor(processor); });
You can also toggle a blur filter on a RemoteVideoTrack as shown below.
room.on('trackSubscribed', track => { if (track.kind === 'video') { const { width, height } = track.dimensions; const processor = new BlurVideoProcessor(width, height, 3); document.getElementById('preview-remote').appendChild(track.attach()); document.getElementById('toggle-blur-remote').onclick = () => track.processor ? track.removeProcessor(processor) : track.addProcessor(processor); } });