Why use grid tokens and joint tokens?

[mimiliaogo]

In your paper, you mention that

It is noteworthy that the grid feature plays an important role to create the united body structure by aligning each point in an appropriate location.

Is there any experiment that can prove this statement? Will there be any significant performance drop if you remove the grid feature?

In addition, since your objective is to predict the human mesh, why do you need joint tokens? Also, why do you use sampling techniques on vertices tokens only but exclude joint tokens?

Thanks.

[kmk3942]

Sorry for the late reply.

1. Unfortunately, we've not conducted any ablation study about the grid feature. But we used this idea following the architecture of previous work, i.e., MeshGraphormer, and they already explained the role of the grid feature. So maybe you can check their paper!

2. As you understand, the goal of our work is to estimate the mesh vertices. But the regression process of joint position can support the estimation of mesh vertices (because the mesh structure and joints are closely related in the human body). Also, we can use additional 2D or 3D pose datasets not including the ground truth vertices during training by computing the loss with predicted joints.
   For the last question, we intended to focus on estimating vertices thus we thought that using our sampling scheme at joint tokens is not necessary.

Hope my answer is helpful.

[herochen7372]

Hello, your task is to generate mesh, but why did you generate keypoints in the end, is it to improve the generation effect of mesh?