-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Empty Volume from a RGBD image #1137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Can you look at the ranges of coordinates of the points and verify that they are within the world coordinates of the volume? |
Hi @bottler, thanks to your comment! I was able to translate the volume so that the world coordinates of the volume fall cover the range of Pointcloud. I also add a sanity check to render the Volume using the same camera but the rendered image is bad. This is the output of the modified script:
This is the modified script:
This is the rendered image using the VolumeRenderer: Can u check why the ray density is so large on the updated volume ? |
Summary: Add option to not rescale the features, giving more control. #1137 Reviewed By: nikhilaravi Differential Revision: D35219577 fbshipit-source-id: cbbb643b91b71bc908cedc6dac0f63f6d1355c85
For those who might still be coming to this issue, the commit referenced above doesn't explicitly solve the large density issue that @phongnhhn92 mentioned in their last comment (which in turn leads to images like the one shared above). As is mentioned in a comment added in that commit pytorch3d/pytorch3d/ops/points_to_volumes.py Lines 255 to 257 in 78fd5af
add_pointclouds_to_volumes are not scaled, and so rescaling is necessary in order to get sensible rendering.
As a running example, I'm using a volume representing a moped. Without rescaling, the max density in the volume after To get something to work a bit better, scale the densities by this max density value:
This gives a render like the following However, it's still unclear how to optimally scale these densities -- in general, using the global max leads to mostly low densities in places which (by most point cloud metrics) would not be low density. Heuristically tuning, you can get better looking results, e.g. here where I scale by 1/5th of the maximum density: in generally, the image displays truer colours and less alpha. However, you can see some pixels are already being affected by high density (clipping to black/white values): Hope this helps someone. I suspect this behaviour is not an "issue", but a fundamental challenge with inferring voxel grid densities from point clouds? It might still be nice to hear from the team on this. |
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days. |
This issue was closed because it has been stalled for 5 days with no activity. |
Hi guys, I have been trying to create a Volume from a RGBD image through the Pointcloud. I have been able to render the image given a PerspectiveCameras but when I try to use the function add_pointclouds_to_volumes, the obtain volume is completely zeros. I have included the code and data in the zip file above.
Volume.zip
The text was updated successfully, but these errors were encountered: