Adapt to animal face 🐱

[ShiJiaying]

Hi，

I’ve made some updates to the project and would like to share. Below are the details of the changes I’ve made:

• Easy to ues LivePortrait on animal face
• X-Pose for keypoints

Here is the Git repository with my updates: LivePortrait

[cleardusk]

Amazing. I have checked the training data again and confirmed that there is no animal data for this released model 🤣

[ShiJiaying]

Amazing. I have checked the training data again and confirmed that there is no animal data for this released model 🤣

Without animal data, the result is quite good!
In my test, the eyes can basically be controlled very well, but human teeth may appear when animal’s mouth opens^ ^ . The implicit keypoints help a lot!

[juntaosun]

@cleardusk

When will the animal model related codes be released? Thank you

[Inferencer]

Amazing. I have checked the training data again and confirmed that there is no animal data for this released model 🤣

Without animal data, the result is quite good! In my test, the eyes can basically be controlled very well, but human teeth may appear when animal’s mouth opens^ ^ . The implicit keypoints help a lot!

If the animals teeth are present on the source photo then the teeth should be correct in theory even on this model. You would have to ensure the driving vids first frame matches the open mouth though which may be cumbersome depending on the dogs mouth openness even if retargeting the first frame of the driving vid.

[mocmocmoc]

but human teeth may appear when animal’s mouth opens

Please post vid, I need to see this 😂

[garbe-github-support]

great work

[Inferencer]

but human teeth may appear when animal’s mouth opens

Please post vid, I need to see this 😂

large-Capuchin-Monkey-photo--d6_trim.mp4

I have an example of this

[ShiJiaying]

but human teeth may appear when animal’s mouth opens

Please post vid, I need to see this 😂

animal6--d0_concat.mp4

It is a badcase of wrong mouth keypoints and no teeth in the source animal pic.

[markyfsun]

It seems that if the character in the source picture has an open mouth, the teeth problem is resolved.
BTW, it can be generalized to humanoid characters like furry.

i--d0_concat.mp4

[cleardusk]

Refer to the updates: https://github.com/KwaiVGI/LivePortrait/blob/main/assets/docs/changelog/2024-08-02.md