Week 4 2021 Fall

Homework:

• Reading:

  • face-api.js — JavaScript API for Face Recognition in the Browser with tensorflow.js
  • Face and hand tracking in the browser with MediaPipe and TensorFlow.js

• Coding:

  • Build an interactive browser experiment on face or hand data
  • Publish your project on GitHub or your own blog, or record a video and put it on your blog. Add your project link below.

See demos live

• Face-api
  • FaceApi_Image_Landmarks
  • FaceApi_Video_Landmarks
• Face Mesh
  • Facemesh_Image
  • Facemesh_Webcam
  • Cat face filter, code
• HandPose
  • Handpose_Image
  • Handpose_Webcam
• Training hand pose model
  • Hand pose tracking + KNN Classification, p5 web editor code, demo video
  • Hand pose tracking + Neural Network, demo video
    • Collect data
    • Train the model
    • Run the model
• Multiple hands detection:
  • Handpose multiple hands with mediapipe If the webcam video is not loading, try open a incognito window, or set the camera access for the browser.
  • KNN Classifier on multiple hands: Change HAND_NUM = 2 if you would like to detect other hand numbers. Once clicking on "Add example" button, it will start collecting data in 5 seconds for 5 seconds.

Resource

• ml5js - faceapi
• ml5js - faceMesh source
• ml5js - handPose source
• tfjs - faceapi-js
• tfjs - faceMesh
• tfjs - handPose

Projects

• LipSync game using Face Mesh model
• Customizable AR face masks - Made with TensorFlow.js
• fingerspelling

Questions?

• add your question here

Add your name / link to your assignment below

• Zairan
• Marcel
• Yilin
• Youming doodleface
• Sara Solfege Hand Sign (ASL) player && Cultural Hand Gestures
• Wendy Show yourself some love
• Nart: Shaking things up with facemesh - Shake your head to scramble your face pixels
• Jenny
• Shaurya : No_ID [ using handpose , the model predicts 'dreaming ' , 'thinking' and no idea(no_id) poses :
  https://editor.p5js.org/ShauryaSeth/sketches/-NmnoUsJU