Export to InstantNGP?

[swtyree]

Hi, I wanted to see how DuST3R poses performed in InstantNGP NeRF reconstruction, and your project seemed to provide a nice shortcut. So I ran InstantNGP's colmap2nerf script on the COLMAP-style output that your scripts produce:

$NGP_PATH/scripts/colmap2nerf.py --images images/ --aabb_scale 4 --text sparse/0/
$NGP_PATH/scripts/run.py transforms.json --gui --train --optimize_extrinsics

The results with InstantNGP are quite a bit worse than the results of Gaussian Splatting with InstantSplat:

I don't see any obvious bugs, but I thought I'd ask if you knew of any reason that this shouldn't work, e.g. are there any known bugs in the COLMAP-style output? Are you doing any tricks to get those files to work for Gaussian Splatting? Thanks for any insights!

[Z-zachary]

Are the camera parameters you used generated by the author method? You turn on the visual camera in training to see if your camera parameters are correct.

I found that the camera parameters generated by the author under the new perspective were wrong, and they were all oriented towards the z axis

[swtyree]

Hi @Z-zachary, I'm not sure what you meant by "camera parameters generated by the author under the new perspective", but I used cameras.txt and images.txt generated by InstantSplat's coarse_init_infer.py ("Dust3r_coarse_geometric_initialization"). Here is a visualization of those camera poses in the InstantNGP GUI. The poses look reasonable, but the reconstruction suggests that they are not accurate.

[Z-zachary]

Ohhhh! i used camera.txt and images.txt generated by InstantSplat's coarse_init_infer.py,but the camera pose is unreasonable.just like:

Could you provide a file that you read the camera's parameters? Thank you so much

[swtyree]

See my first message. I used the colmap2nerf.py script from InstantNGP to convert the COLMAP files (cameras.txt and images.txt) in sparse/0/ into a transforms file:

$NGP_PATH/scripts/colmap2nerf.py --images images/ --aabb_scale 4 --text sparse/0/