How to make gen_pose_map_cano_smplx.py

[IceFtv]

Very nice work! I tried using smplx to train my own videos, but I found that the gen_pose_map_cano_smplx.py is missing. So, I tried to mimic the model's call to make this file, but I failed. Can you tell me how to make this file or provide it? Thank you.
Here is the code I imitated：

smpl_model = smplx.SMPLX(model_path='../assets/smpl_files/smplx', use_pca=False, num_pca_comps=45, flat_hand_mean=True, batch_size=1)
cano_smpl = smpl_model.forward(betas = smpl_data['beta'],
global_orient = smplx_cpose_param[:, :3],
transl = torch.tensor([[0, 0.30, 0]]),
body_pose = smplx_cpose_param[:, 3:66],
jaw_pose = smplx_cpose_param[:, 66:69],
leye_pose=smplx_cpose_param[:, 69:72],
reye_pose=smplx_cpose_param[:, 72:75],
left_hand_pose= smplx_cpose_param[:, 75:120],
right_hand_pose= smplx_cpose_param[:, 120:])

However, doing so would result in errors in the body_model.py or lbs.py files. Could you tell me where the problem is? I would greatly appreciate it if you could reply.

[superhero177]

Hello, have you solved this problem?

[IceFtv]

Hi. @superhero177 I found that the batch_size of "betas" in gen_pose_map_cano_smpl is only 1, while "betas" in smplx I provided are not 1. Therefore, I used the first frame's "betas" or the average "betas" as input for smpl_model, which may cause some issues. If you have a better solution, you can share it.

[IceFtv]

"betas": results["smpl_betas"].mean(axis=0),
I just discovered that this is done in the pre-processing file of InstantAvatar, which may be a better option to get mean.

[superhero177]

ooh！Thank you, I think this is a good approach.

[huliangxiao]

Hi, sorry for the delayed response. It would help if you modified the function 'save_obj' in gen_pose_map_cano_smpl.py to save the corresponding SMPL-X obj and set uv_template_fn=the path of SMPL-X template.
This is the save_obj code for SMPLX-X:

smpl_data = torch.load( data_path + '/smpl_parms.pth')

smpl_model = smplx.SMPLX(model_path='./assets/smpl_files/smplx',gender = 'neutral', use_pca=False, num_pca_comps=45, flat_hand_mean=True, batch_size=1)

cano_dir = os.path.join(data_path,)
live_smpl = smpl_model.forward(betas = smpl_data['beta'][0][None],
                            transl = torch.tensor([[0, 0.35, 0]]),
                            global_orient=smplx_cpose_param[:, :3],
                            body_pose=smplx_cpose_param[:, 3:66],
                            jaw_pose=smplx_cpose_param[:, 66:69],
                            leye_pose=smplx_cpose_param[:, 69:72],
                            reye_pose=smplx_cpose_param[:, 72:75],
                            left_hand_pose=smplx_cpose_param[:, 75:120],
                            right_hand_pose=smplx_cpose_param[:, 120:])

ori_vertices = live_smpl.vertices.detach().cpu().numpy().squeeze()
joint_mat = live_smpl.A
print(joint_mat.shape)
torch.save(joint_mat ,join(cano_dir, 'smplx_cano_joint_mat.pth'))

mesh = trimesh.Trimesh(ori_vertices, smpl_model.faces, process=False)
mesh.export('%s/%s.obj' % (cano_dir, 'cano_smplx'))

[IceFtv]

Thanks for your reply. But I am still a bit confused about the role of the second stage training. I noticed that other issues mentioned that the role of the second stage training is not obvious. Could you tell me the significance of two-stage training.