HoloSkype

FaceID to Hologram all-in-one script

Hardware Requirements

• iPhone/iPad with FaceID camera and Record3d app with unlocked USB streaming (In-app purchase)
• Looking Glass holographic display

Software Requirements

Server

• Python 3 with following modules opencv-python, scikit-image, record3d, diff_match_patch

Client

• [Optional] HoloPlay service (HoloSkype is using a patched version of HoloPlay.js to allow HoloPlay-less operation.)
• Web browser (If your client is an iOS device, use this custom browser)

See it in action here and here.

Setup

• If you plan on using HoloSkype without the HoloPlay service with hardcoded calibration values, get yours here and use them to replace my values on row 51.
• If you are using a single computer as both server and client, you might have to reduce FPS on row 36. I recommend using a PC as a server and iPad as a client for best results.

To-do

• Initial release
• LiDAR sensor support
• Texture effects (Edge/Jet)
• WebKit compatibility

Inspiration

I saw this demo made with Unity and I decided to re-create it using open web technologies. This demo then inspired me to add the edge effect.

References:

1. Record3d Python Demo App [link]
2. Stack Overflow answer on MJEPG streamer [link]
3. Stack Overflow answer on turning depth map into a point cloud with Three.js [link]
4. Thomas Le Coz's HoloPlay.js demo [link]