RGB loss not affecting shape + tech report experiment question

[sjcv]

1. For the task of single image 3d reconstruction(using 2d supervision), as outlined in the tech report of Pytorch 3d, did you train a single model for all the ShapeNet categories or separate models for different categories?

2. In the fit_textured_mesh tutorial, I observe that the RGB loss only affects the texture and not the shape. Is that expected? There will be cases when the texture of a 2d object image provides strong signals for shape, which silhouette doesn't, especially when we have limited silhouettes of an instance.

3. Would really help if you can provide the code for the experiment mentioned in (1). Thanks!

[gkioxari]

@sjcv
Re 1. We train one single model for all categories. We follow the training recipe of Mesh R-CNN.

Re 2. The RGB loss should affect both texture and shape. The gradients flow back to both vertex rgb and coordinates predictions. Can you specify exactly how you come to the conclusion that the RGB loss only affects the texture?

Re 3. It is in our TODO list and we will get to it.

[sjcv]

For the RGB loss, I came to this observation on the basis of qualitative evaluation on custom dataset, and thus wanted to confirm if it is expected behaviour or not. Thanks for clarifying.

Look forward to the code. Thanks!