Video distribution

When making room or application calls, number of incoming video streams is limited, which are prioritized based on the user activity. This document describes limits and processes which are used for video distribution.

When sending camera video, a technology called simulcast is used, which sends up to 3 layers of different resolution and quality to our servers: high (720p), medium (360p) and low (180p). Screen share is always sent at the highest quality ( 1080p). Our backend then determines whether to forward the stream to the other users and in which quality. This process takes into the account user activity (which uses volume detection), video mode (configurable as part of the call options) and user platform.

There are two video modes currently available:

• Presentation mode - video mode which enables subscribing to a limited number of video streams
• Grid mode - video mode which enables subscribing to all video streams

The number of videos in each mode and their quality is determined by the SDK platform:

• Presentation
  • when there is no active screen share
    • 3 (web) or 1 (mobile) camera stream(s) distributed in high quality
    • Additional 3 (web) or 0 (mobile) camera stream(s) distributed in low quality
    • All streams are ordered based on user activity
  • when there is an active screen share
    • Only one screen share can be active at the time (if another user requests screen sharing, currently active screen share is removed and replaced by a new one)
    • Screen share stream distributed in the highest quality
    • Screen sharing user's camera stream and additional 3 (web) or 1 (mobile) camera streams distributed in low quality
    • Screen sharing user's screen share and camera streams are pinned
    • Additional camera streams are ordered based on user activity
• Grid
  • When there is only 1 stream, it is distributed in high quality
  • When there are up to 3 streams, they are distributed in medium quality
  • Otherwise, all streams are distributed in low quality
  • All streams are ordered based on user activity
  • Only one screen share can be active at the time (if another user requests screen sharing, currently active screen share is removed and replaced by a new one)

Ordering streams based on user activity is done by using volume detection. If user's microphone volume exceeds certain threshold, the user is considered active/talking, thus participant-started-talking event is sent and it's video feeds get prioritized (which can change which video streams other users are receiving). Once user's microphone volume dips below the threshold, user is no longer considered active/talking, so participant-stopped-talking event is sent.

Changes in video subscriptions are automatically handled by the SDK. User will receive information on streams via the participant-camera-video-added, participant-camera-video-removed, participant-screenshare-added and participant-screenshare-removed events.