Synthetic NeRF dataset creation tool

This tool was created to leverage the power of AI葵 (kwea123)'s Lightning fast NeRF NGP_PL. It uses the colmap2nerf script from NVIDIA's Instant NeRF implementation and COLMAP to produce a Synthetic NeRF dataset formated like those in Facebook Research NSVF Synthetic NeRF datasets.

Feel free to use and enhance. I wrote this in a few free hours and some of the code might not be pythonic enough. You are more than welcome to contribute to this effort.

📖 Dataset structure

<dataset_name>
|-- bbox.txt         # bounding-box file
|-- intrinsics.txt   # 4x4 camera intrinsics
|-- rgb
    |-- 0.png        # target image for each view
    |-- 1.png
    ...
|-- pose
    |-- 0.txt        # camera pose for each view (4x4 matrices)
    |-- 1.txt
    ...

The resulting bbox.txt file contains a line describing the initial bounding box and voxel size:

x_min = -7.500 y_min = -7.500 z_min = -7.500 x_max = 8.500 y_max = 8.500 z_max = 8.500 initial_voxel_size = 0.4

I hardcoded those values by referencing NVIDIA's Instant NeRF's rendering GUI. You are more than welcome to change them to fit your needs.

The datasets are split with view indices. For example, "train (0..100), valid (100..200) and test (200..400)" mean the first 100 views for training, 100-199th views for validation, and 200-399th views for testing.

Required libraries

tqdm, halo, Pillow, json, opencv-python, argparse, pathlib, numpy

💻 Usage

• Have Colmap installed in your system and added to your system PATH.
  
• Create an images folder and add the photos you want for the dataset.
  
• Run python dataset.py
  
• The tool will convert .JPG files to .PNG format and -if needed-, resize to a standard 800 x 800. This is done for ease of use, faster COLMAP processing and avoid potential errors when training.
  
• At the end of processing, you should now have an NSVF style formatted dataset to train with NGP_PL.
  

📺 The video below shows the tool in action

dataset.mp4

❕ Changelog

• Standarized image output and size.
• Added randomization to test and validation sets.
• Removed redundant code.
• Added function to rezise images if they aren't 800 x 800 only.
• Added functions to exit gracefully on the dataset script.

🔨 To Do

• Create GUI
• Add funtionality to prepare datasets for colorless and colorized mesh extraction

👍 Acknowledgments

AI葵, kwea123 for the amazing work in the instant NeRF field.
NVIDIA Research Projects for their breakthroughs in this field.
Meta -Formerly Facebook- Research