Skip to content

Latest commit

 

History

History
55 lines (43 loc) · 2.14 KB

README.md

File metadata and controls

55 lines (43 loc) · 2.14 KB
Raw

Virtual Background

Inspiration from: https://elder.dev/posts/open-source-virtual-background/

Overview of Steps to reproduce

  1. Read the camera with cv2 - OpenCV python bindings
  2. Use body-pix tensorflow model to create a mask of your body on the frame blogpost. Use tfjs-node or tfjs-node-gpu if you have a graphic card supporting CUDA
  3. Combine the background and foreground, using the mask and add holo or other visual effects to the foreground
  4. Outputting Video: use
    • v4l2loopback - to create a virtual webcam device

      On Ubuntu 18.04 I needed to build v4l2loopback from the source code to use the latest version 0.12

    • pyfakewebcam to write RGB frames to a fake webcam

Install

  • Body-pix: cd bodypix && npm install

  • Fake-cam: cd fakecam && pip3 install --no-cache-dir -r ./requirements.txt

  • Install v4l2loopback

    • Either

      sudo apt install v4l2loopback-dkms

    • In my case (Ubuntu 18.04)

      git clone https://github.com/umlaeute/v4l2loopback.git --depth=1
cd v4l2loopback
sudo make install
sudo depmod -a

  • Configure fake webcam device with v4l2loopback

    # Reset allvirtual devices
sudo modprobe -r v4l2loopback
# Create virtual /dev/video20 with exclusive_cap (required for chrome)
sudo modprobe v4l2loopback devices=1 video_nr=20 card_label="v4l2loopback" exclusive_caps=1
# Check if virtual cam created
v4l2-ctl --list-devices

Run

  • cd bodypix && npm start
  • cd fakecam && python3 fake.py
  • ffplay /dev/video20 for debug or use with an online chat

Ideas

  • try using facemesh and replace face with something else
  • experiment with JACK Rack and audio/mic tuning tutorial

Links