Portrait-Talker

Talking head animation

Note: This project is not complete it is still WIP.

Installation

step 1. Clone repo

git clone https://github.com/harisreedhar/Portrait-Talker.git
cd Portrait-Talker

step 2. Download pre-trained models

put these models in Portrait-Talker/pretrained_models

• det_10g.onnx
• vox512.pt
• shape_predictor_68_face_landmarks.dat
• wav2vecDS.pt
• clip_training_DINet_256mouth.pth

step 3. Create env & Install dependencies

conda create -n portrait-talker python=3.10 -y
conda activate portrait-talker
conda install pytorch==2.0.0 torchvision==0.15.0 torchaudio==2.0.0 pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt

step 4. Test Run

python run_video_driven.py  --source_path ./test/source.jpg --driving_path ./test/driving.mov

or

python run_audio_driven.py  --source_path ./test/source.jpg --driving_path ./test/driving.wav

Acknowledgment

• LIA
• DINet
• Insightface
• Dlib