Small holes when trying to render mesh (z-fighting?)

[tomer-grin]

🐛 Bugs / Unexpected behaviors

I am getting the following image when trying to render the object.

Might it be related to this issue?
#283

I am using the next piece of code to define the renderer:
`blend_params = BlendParams(sigma=blend_sigma, gamma=blend_gamma, background_color=(0.0, 0.0, 0.0))

    # Define the settings for rasterization and shading. Here we set the output image to be of size
    # 256x256. To form the blended image we use 100 faces for each pixel. We also set bin_size and max_faces_per_bin to None which ensure that 
    # the faster coarse-to-fine rasterization method is used. Refer to rasterize_o3d_to_meshes.py for 
    # explanations of these parameters. Refer to docs/notes/renderer.md for an explanation of 
    # the difference between naive and coarse-to-fine rasterization. 
    raster_settings = RasterizationSettings(
        cull_backfaces=cull_backfaces,
        image_size=image_size, 
        perspective_correct=False,
        max_faces_per_bin=None,
        bin_size=None,
        blur_radius=blur_coef*np.log(1. / 1e-4 - 1.) * blend_params.sigma,
        faces_per_pixel=faces_per_pixel
    )

    lights = PointLights(device=self.device, location=((5, -10, -50),),diffuse_color=((0, 0, 0),),
                specular_color=((0, 0, 0),),)
    shader_params = dict(device=self.device, cameras=self.cameras_py3d, lights=lights)

    if renderer_type.lower() == 'silhouette':
        shader=SoftSilhouetteShader(blend_params=blend_params)
    elif renderer_type.lower() == 'hard_gouraud':
        shader=HardGouraudShader(**shader_params)

    elif renderer_type.lower() == 'soft_gouraud':
        shader=SoftGouraudShader(**shader_params)

    elif renderer_type.lower() == 'soft_phong':
        shader=SoftPhongShader(**shader_params)
    elif renderer_type.lower() == 'simple_shader':
        shader=SimpleShader(blend_params=blend_params, hard_or_soft=hard_or_soft)
    else: 
        shader=HardPhongShader(**shader_params)


    # Silhouette renderer 
    renderer_silhouette = MeshRenderer(
        rasterizer=MeshRasterizer(
            cameras=self.cameras_py3d, 
            raster_settings=raster_settings
        ),
        shader=shader
    )

`

[bottler]

It's not related to #283, which is definitely fixed in all releases since 0.2.5. Can you try again with bin_size=0 (which forces naive rasterization)?

[tomer-grin]

@bottler
Much better,

Thanks a lot Jeremy!
But the forward pass is painfully slow as explained in: https://pytorch3d.readthedocs.io/en/latest/modules/renderer/rasterizer.html#pytorch3d.renderer.mesh.rasterizer.RasterizationSettings

Any lead on this matter?
And I would be eager to know how did you come up with such a good intuition about this issue,
as well as reading materials related to solving these issues (computer graphics related to pytorch3d renderer)
Will openGL resources work?

[bottler]

We can assume the problem is bin overflow as described in #348. We've seen this a few times, but the symptom has usually been square holes. You should be able to fix it in a faster way by setting max_faces_per_bin in the RasterizationSettings to something larger than the default, while setting bin_size back to None. This issue is a bit specific to the way PyTorch3D works with CUDA, and general material wouldn't help. PyTorch3D guesses the size required for the bins before rasterization, and doesn't recover gracefully when that size was underestimated.

It is not obvious to us how to fix this issue (so that the rasterizer can run without needing an estimate or could rerun after discovering an underestimate) without adding synchronization calls which would slow down rasterization in the regular case. It is even not obvious how to emit a warning to the user when this situation has occurred without slowing down the normal case. (The condition is detected only within a CUDA kernel. We could use CUDA's printf but that wouldn't help notebook users.)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.