-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Explain about parameters of Candide face model #2
Comments
Hi, The content of candide.npz is basically a processed version of the original candide model you posted in your link.
If you have any other questions, or if that was not clear, please let me know. Marek |
Thank you so much for answering me. I used this program to swap my face with a picture, but I noticed that when my eyes close, the eyes of swapped face are not closed. Thought of the reason is that face model lacked of Shape Units, so that I added Shape Units parameters to optimize the accuracy of face model when fitting it in landmark points. But even when did that the eyes are still not closed. Do you think the reason of this is related to when fitting Candide face model or swapping faces? And how to fix that? |
Hi, Congratulations on converting the app to C++!
What I would suggest is that you try removing the triangles of the mesh that are located in the eye region so there are holes in those areas of the model. If you do that the swapped face will show the user's real eyes as it does with the mouth. This might prove more realistic than the current effect. Let me know what you did and how it went. Marek |
Thank you, I will try your suggestions and tell you what I found. |
@vudangthinh Great work converting the code base into C++. Are you planning to open source the implementation in c++? We intend to do something similar. We would like to build atop of your work instead of re-inventing the wheel! How does that sound to you? @MarekKowalski Apologies for hijacking the thread :( No private messaging on Github yet. |
@vudangthinh Did you add any new shape units that would close the eyelids? If not, that might be the issue., I'm not sure if the original model had any. If you did, make sure that the vertices in those are attached to the 2D landmarks. |
@abhilashi Sorry, I cannot open source it. However, I think with the source by Marek, you can easy to convert it to C++. If you have any private messages, you can email to me thinhbn2408@gmail.com @MarekKowalski When adding shape units, the eyelids close a little bit, but it's not absolutely close. |
@tvd2840 hello,when convert the app to C++, do you use opengl to subsitute the pyOpenGL and pygame related part? |
Yeah, it boosts the speed of swapping. |
@MarekKowalski hi ,from your answer,i got that you make some blendshapes by changing shape unit parameters or action units parameters in candide , and then using |
Hello! Could you please share the C++ implementation? |
@xiexiexxs Hi, I convert the shape and action units into blendshapes, which are S_i in the equation that you linked. As for the fitting function, yes that is the one you linked. The dimensionality of the blendshapes is the same as S_0, I don't really remember what it is. You can check it in the code however, the name of the variable is meanShape3D or something like that. @DmitryBorin Hi, one of my other repositories https://github.com/MarekKowalski/HoloFace implements a similar functionality in C++. Look at the Fit3D file under the link below: Marek |
@MarekKowalski Thank you for your answer! I watched this implementation, but I would like to test it on Linux. So, maybe you have an implementation of the basic functionality (fitting with candide, find rotation/translation, etc) without special Windows libs? |
Hi, I do not have any other implementations. I am however pretty sure that none of the model fitting code actually requires windows-only libraries. Marek |
@MarekKowalski also you said "blendshapes: blendshapes are shapes you can add to the Candide neutral state face to modify it. An example of a blendshape would be a smile or an eye brow raise. The blendshapes are derived from the Animation Units defined in Candide." i am confused about how are the blendshapes from? |
Hi, Below I am pasting a link to a very "dirty" script that does the conversion to blendshapes, so you can check it out yourself. Best regards Marek |
candide.py open candide.txt? what is candide.txt? |
Candide.txt is a file describing the CANDIDE face model. Yes, the Python script opens that file. It also processes it into an .npz file. |
hello, sorry for asking how can i add some blendshapes by myself, such as fit the glasses, is there some example? |
In order to add glasses you would have to add more than just blendshapes. You would have to add them to the mesh. You could do this by saving the mesh to a format than can be read by some 3D modelling software (like Blender) and editing it there. You would then need to be able to load the mesh back into the app. |
thank you very much for you reply! |
I saw that on the function you write to read parameters from candide.npz
It is seem that content of file candide.npz is different from candide file from http://www.icg.isy.liu.se/candide/candide3.wfm
And can you explain the meaning of
mean3DShape
,mesh
,idxs3D
,idxs2D
,blendshapes
array?The text was updated successfully, but these errors were encountered: