How to use diffrent obj model?

[AdamMayor2018]

Incredible work！Thanks!
I have a question on using diffrent obj model.
I tried to use obj model file created by deca, but meet a error:

(meshtalk) ubuntu@ubuntu-X640-G30:/data/cx/GANs/meshtalk$ python animate_face.py --model_dir weights/pretrained_models --audio_file test.wav --output outputs --face_template myasset/mzd.obj
/home/ubuntu/.local/lib/python3.8/site-packages/torchaudio/backend/utils.py:53: UserWarning: "sox" backend is being deprecated. The default backend will be changed to "sox_io" backend in 0.8.0 and "sox" backend will be removed in 0.9.0. Please migrate to "sox_io" backend. Please refer to pytorch/audio#903 for the detail.
warnings.warn(
load assets...
load models...
Loaded: weights/pretrained_models/vertex_unet.pkl
Loaded: weights/pretrained_models/context_model.pkl
Loaded: weights/pretrained_models/encoder.pkl
animate face mesh...
/home/ubuntu/.local/lib/python3.8/site-packages/torch/functional.py:515: UserWarning: stft will require the return_complex parameter be explicitly specified in a future PyTorch release. Use return_complex=False to preserve the current behavior or return_complex=True to return a complex output. (Triggered internally at /pytorch/aten/src/ATen/native/SpectralOps.cpp:653.)
return _VF.stft(input, n_fft, hop_length, win_length, window, # type: ignore
/home/ubuntu/.local/lib/python3.8/site-packages/torch/functional.py:515: UserWarning: The function torch.rfft is deprecated and will be removed in a future PyTorch release. Use the new torch.fft module functions, instead, by importing torch.fft and calling torch.fft.fft or torch.fft.rfft. (Triggered internally at /pytorch/aten/src/ATen/native/SpectralOps.cpp:590.)
return _VF.stft(input, n_fft, hop_length, win_length, window, # type: ignore
Traceback (most recent call last):
File "animate_face.py", line 93, in
geom = template_verts.cuda().view(1, 1, 6172, 3).expand(-1, T, -1, -1).contiguous()
RuntimeError: shape '[1, 1, 6172, 3]' is invalid for input of size 15069

What should I do if I want to animate different obj files?

[AdamMayor2018]

I created my obj model with DECA, Then used houdini to add more verticles to 6172.
After that I put my model into meshtalk, the animation result looks bad.

[alexanderrichard]

It's not just about having the same number of vertices - the vertices need to represent the same facial positions. In other words, your mesh will not be compatible with the meshes used to train this system. For example, your vertex with index 1 might be on the nose, but the meshtalk vertex with index 1 might be on the ear.

[AdamMayor2018]

I see, is there a way to animate custom 3D face models ?

[alexanderrichard]

You can try to align your face mesh to the provided mesh. If you manually scale your mesh to the same size and orientation as the provided mesh, ICP between the surface of your mesh and the vertices of the provided mesh should give you a good enough alignment.