need guidance on interior scene using custom data with MonoSDF

[DavidNP96]

Hi all,
I am trying to create an indoor scene, however, viewer results remain blurry (as per the image below). When I am importing the object into unreal I get a blob not even closely resemble the interior scene. I feel like I might be doing multiple things wrong in creating this result maybe someone here can help?

steps to recreate:

1. see data here https://drive.google.com/drive/folders/1EUspzF3vwK18N6kOoe4Cj6x3dG6JxZbB?usp=sharing
   run following commands:
2. ns-process-data images --data data/sdfstudio-demo-data/${image_map_naam}/ --output-dir data/sdfstudio-demo-
   data/{image_map_naam}/
3. python scripts/datasets/process_nerfstudio_to_sdfstudio.py --data data/sdfstudio-demo-data/${image_folder_name}/ --output-dir data/sdfstudio-demo-data/${image_folder_name}/processed --scene-type indoor --data-type colmap --mono-prior --crop-mult 1 --omnidata-path ./omnidata/omnidata_tools/torch --pretrained-models ./pretrained_models
4. ns-train monosdf --pipeline.model.sdf-field.inside-outside True --pipeline.model.mono-depth-loss-mult 0.1 --pipeline.model.mono-normal-loss-mult 0.05 --vis viewer --experiment-name neus-facto-replica1 sdfstudio-data --data data/sdfstudio-demo-data/conf/processed --include_mono_prior True

results in viewer:

results in unreal:

[niujinshuchong]

@DavidNP96 Your dataset looks difficult. Could you get some reasonable results with nerfacto?

[DavidNP96]

thank you for your response @niujinshuchong the result of nerfacto is quite alright. However, there are quite a number of artifacts.

nerfacto_results.mp4

[niujinshuchong]

@DavidNP96 Nerfacto's result looks good. Maybe you could try to train a volsdf/neus model without monocular prior first to check how the results look like before adding monocular prior.

[DavidNP96]

@niujinshuchong

This is the nerf result when extracting its mesh (using poisson meshing):

nerf_with_poisson.mp4

This is volSDF result without monocular priors :

blender_Mcq7dLYolb.mp4

These are the parameters I used for volSDF: ns-train volsdf --pipeline.model.sdf-field.inside-outside True --vis viewer

[niujinshuchong]

@DavidNP96 Looks like it's a difficult dataset. Maybe you could try other method in the repo, e.g. bakedsdf and bakedsdf-mlp. But I am not sure whether it will produce reasonable result or not.