Face API / Face Matching with ChromaDB

This module provides the basics for identifying and recognizing faces using the Face API. We will leverage ChromaDB as well.

Docker Container Setup

You will need to install some more packages to use TensorFlow for face detection.

• apt update
• apt install -y libhdf5-serial-dev pkg-config
• pip install h5py
• pip install tensorflow

Then run npm install from this directory.

Then:

• npm rebuild @tensorflow/tfjs-node --build-from-source
• bash ./download-models.sh

Images Download

The download-models.sh script should download a .zip file with IMDB images. This was assembled from two sources:

• names.csv, which is the original list of actor names from the homeworks. You can update this with whatever IMDB actors you have.
• IMDB-Face.csv, from https://github.com/fwang91/IMDb-Face.

Where we ran python get-images.py to join the overlapping nconsts and downloaded associated images. If you have a larger subset of names from IMDB, you can re-run this step yourself and get a lot more images!

Creating Your ChromaDB Instance

We will use ChromaDB for a very simple purpose: we'll index (as keys) embeddings for (as values) the contents of files (whose names will be associated with the embeddings as ChromaDB metadata). Given an embedding $e_1$ (computed from a function below!) we can search for $(e,v)$ pairs where $e$ is similar to $e_1$ and $v$ represents the metadata associated with $e$. In that metadata will be a filename.

Launch ChromaDB in Docker, via chroma run --host 0.0.0.0.

In a separate Docker Terminal, from the same directory, run node app.js and let it index everything!

Useful functions

Take a look at the functions in app.js, to see examples of computing an embedding from an image, opening a connection to Chroma, indexing in Chroma, and looking up an entry in Chroma.

• initializeFaceModels sets up the Face-API (and TensorFlow-JS) to compute embeddings
• getEmbeddings uses the Node Face-API to find faces within an image file and return a list of embeddings
• compareImages uses the Node Face-API to compare the faces within two image files
• indexAllFaces takes an image, finds all faces and computes their embeddings, and puts these into ChromaDB
• findTopKMatches uses ChromaDB to find the most similar embedding to that of a supplied image