Easily add virtual background effects to your video/camera input inside any web browser

Try a working demo here

Install from npm using:

npm install virtual-bg

or include directly in HTML:

<script src="https://cdn.jsdelivr.net/npm/virtual-bg@latest/virtual-bg.min.js"></script>

A simple example to apply blur/image background to camera input:

index.html

<div>
    <video autoplay="true" id="inputVideoElement"></video>
    <canvas id="output_canvas"></canvas>
</div>

index.js

import { segmentBackground, applyBlur, applyImageBackground } from'virtual-bg';

const inputVideoElement = document.querySelector('#inputVideoElement');
const outputCanvasElement = document.querySelector('#output_canvas');

let myStream = await navigator.mediaDevices.getUserMedia({
      video: true
});

inputVideoElement.srcObject = myStream;

// segments foreground & background
segmentBackground(inputVideoElement, outputCanvasElement);  

// applies a blur intensity of 7px to the background 
applyBlur(7); 

// applies an image background
const image = new Image();
image.src = 'https://imageurl.jpg'
applyImageBackground(image);

API:

To separate foreground from background:

segmentBackground(inputVideoElement, outputCanvasElement, modelSelection <optional>); 

Parameters:

• inputVideoElement: <HTMLVideoElement> Source of video which is to be used for applying segmentation.
• outputCanvasElement: <HTMLCanvasElement> Canvas Element where output is to be displayed.
• modelSelection: Numerical value to select type of model to be used for segmentation.
  0 : General
  1 : Landscape (default)
  General (0) can be used when there is only 1 person in the view and not far away from the camera.
  Landscape works better if there are multiple people or distance from the camera is more.

To apply blur effect to background:

applyBlur(blurIntensity <optional>); 

Parameters:

• blurIntensity: Numerical value in pixels to apply a blur filter to the background. This uses the same filter as used in CSS blur filter
  default = 7

To apply image to background:

applyImageBackground(imageElement); 

Parameters:

• imageElement: <HTMLImageElement> Image element to be set as background. You can create a new element dynamically using the Image() constructor.

To apply video to background:

applyVideoBackground(videoElement); 

Parameters:

• videoElement: <HTMLVideoElement> Video element to be set as background.

To apply screen capture to background:

applyScreenBackground(screenCaptureStream); 

Parameters:

• screenCaptureStream: <MediaStream> Stream obtained from the Screen Capture API

To change foreground type. Can be used to set foreground as a smaller snip at the bottom right corner of the output like a YouTube/Twitch stream.

An example use case for this can be when user has set screen capture as the background and want to present something.

changeForegroundType(foregroundType, offset<optional>); 

Parameters:

• foregroundType: normal | presenter Type of foreground. Presenter mode shrinks the foreground and displays it in the bottom right corner.
• offset: Numerical value in pixels for setting custom offset when presenter mode is selected.
  For example: -100 offset will move the foreground by 100 pixels to the right.

Note: I'm using mediapipe's selfie segmentation model and library as a base.

For any doubts/feedback/suggestions, please start an issue/discussion at https://github.com/akhil-rana/virtual-bg or contact me at contact@akhilrana.com