Convert SMPL-H to SMPL-X

[Jiankai-Sun]

Hi, thanks for the code! I tried to convert an SMPL-H model (.obj generated by the following code piece) to an SMPL-X representation. While trying to run this code python -m transfer_model --exp-cfg config_files/smplh2smplx.yaml, I met several problems.

• .obj generation code using trimesh.Trimesh

seq = np.load(sample_paths[seq_no])
BATCH_SIZE = seq['poses'].shape[0]
smplm, smplf, _ = load_models(True, batch_size=BATCH_SIZE, model_type=args.model_type)
thetas = torch.from_numpy(seq['poses'][:BATCH_SIZE, 3:66]).float()
if MODEL_TYPE == 'smpl':
    thetas = torch.cat((thetas, torch.zeros(BATCH_SIZE, 1 * 3 * 2)), dim=1)
elif MODEL_TYPE in ['smplx', 'smplh']:
    thetas = torch.cat((thetas, torch.zeros(BATCH_SIZE, 1 * 3 * 0 + 1 * 3 * 0)), dim=1)
pose_hand = torch.from_numpy(seq['poses'][:BATCH_SIZE, 66:]).float()
global_orient = torch.from_numpy(seq['poses'][:BATCH_SIZE, :3]).float()
# get betas
betas = torch.from_numpy(seq['betas'][:args.num_betas]).float().unsqueeze(0).repeat(thetas.shape[0], 1)
# get root translation
trans = torch.from_numpy(seq['trans'][:BATCH_SIZE]).float()
subject_gender = seq['gender']
# SMPL forward with gender specific model
if str(seq['gender']).lower() == 'male':
    SMPLOutput = smplm.forward(transl=trans,
                               global_orient=global_orient,
                               hand_pose=pose_hand,
                               body_pose=thetas,
                               betas=betas,
                               pose2rot=True, )
    faces = smplm.faces
elif str(seq['gender']).lower() == 'female':
    SMPLOutput = smplf.forward(transl=trans,
                               global_orient=global_orient,
                               body_pose=thetas,
                               hand_pose=pose_hand,
                               betas=betas,
                               pose2rot=True, )
    faces = smplf.faces
else:
    SMPLOutput = smplm.forward(transl=trans,
                               global_orient=global_orient,
                               body_pose=thetas,
                               hand_pose=pose_hand,
                               betas=betas,
                               pose2rot=True, )
    faces = smplm.faces
all_step_vertices = SMPLOutput.vertices.to(DEVICE).float()  # torch.Size([100, 3])
print('SMPLOutput.vertices.shape: ', all_step_vertices.shape)

save_folder = '/'.join(sample_paths[seq_no].split('/')[3:-1])
obj_name = sample_paths[seq_no].split('/')[-1]
for i in range(all_step_vertices.shape[0]):
    if args.model_type in ['smpl', 'smplh']:
        mesh = trimesh.Trimesh(vertices=c2c(all_step_vertices[i]), faces=faces,
                               vertex_colors=np.tile(colors['grey'], (6890, 1)))
    else:
        mesh = trimesh.Trimesh(vertices=c2c(all_step_vertices[i]), faces=faces, vertex_colors=np.tile(colors['grey'], (10475, 1)))
    os.makedirs(osp.join(OUTPUT_DIR, save_folder), exist_ok=True)
    save_dir = osp.join(OUTPUT_DIR, save_folder, '{0}_{1:02d}.obj'.format(obj_name, i))
    save_mesh(mesh, None, save_dir)
    print('Saved to {}'.format(save_dir))
    obj_list.append(save_dir)

• smplh2smplx.yaml

datasets:
    mesh_folder:
        data_folder: 'transfer_data/meshes/smplh'
deformation_transfer_path: 'transfer_data/smplh2smplx_deftrafo_setup.pkl'
# mask_ids_fname: 'smplx_mask_ids.npy'
mask_ids_fname: 'transfer_data/smplx_mask_ids.npy'
summary_steps: 100

edge_fitting:
    per_part: False

optim:
    type: 'trust-ncg'
    maxiters: 100
    gtol: 1e-06

body_model:
    model_type: "smplx"
    gender: "neutral"
#    folder: "transfer_data/body_models"
    folder: '../Dataset/amass_contacts/models/smplx/SMPLX_MALE.npz'
    use_compressed: False
    use_face_contour: True
    smplx:
        betas:
            num: 10
        expression:
            num: 10

Input:

model_transfer Output:

Question: I suspect the input .obj file is incorrect, even though the input “looks” correct. Is there any requirements for the transfer_model input?

Thank you for taking the time to solve our problems in advance!

Best

[kbrodt]

I have the same issue. For some poses of SMPL meshes I get weird result.

[Luciano07]

I had the same issue, this is because trimesh changes the vertices values by default.
add the flag "process=False" when you load the mesh:

mesh = trimesh.Trimesh(vertices=c2c(all_step_vertices[i]), faces=faces, vertex_colors=np.tile(colors['grey'], (10475, 1)),process=False)

After that, should work!

[gngdb]

I spent days debugging this, the authors responded and this was the same problem. I wish I'd seen this issue.