RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction

[Project Page] [ArXiv]
Isaac Kasahara, Shubham Agrawal, Selim Engin, Nikhil Chavan-Dafle, Shuran Song, Volkan Isler
Samsung AI Center, New York

Table of contents:

1. Setup environment
2. Inference
3. Citation

Setup environment

1. Make a new directory

   mkdir RIC_Project
   cd RIC_Project

2. Install depth completion repo from ClearGrasp in this folder

   Follow steps 1, 2, and 4 on their page. (You do not need their data or models)

3. Clone our repo inside RIC_Project folder

   Navigate back to RIC_Project folder

   git clone https://github.com/SamsungLabs/RIC.git

   Folder structure should look like this:

   RIC_Project                # Root folder
   ├── RIC                    # Code for RIC (This Repo)
   └── ClearGrasp             # Code for depth completion (ClearGrasp)

4. Install our python dependencies

   cd RIC
   python3.8 -m venv venv
   source venv/bin/activate
   pip install -r requirements.txt

5. Download weights for depth completion

   Download the weights for surface normal and occlusion boundary estimation and place them in the RIC/weights/ folder.

6. Install HOPE Dataset

   Navigate to the HOPE dataset page and follow the download instructions.

7. Add OpenAI API Key

   If you don't have one already, follow these instructions:

   • First make an OpenAI account if you don't have one already
   • Next go here https://platform.openai.com/api-keys
   • Click on create new secret key
   • Copy the key

   Once you have the key, paste the key on line 28 of "inference_hope_dataset.py"

Inference

1. To generate predictions:

   python3 inference_hope_dataset.py --data_dir "path/to/hope-dataset/hope_video/" --write_file_root "results/"

2. To evaluate predictions:

   python3 evaluate_results.py --results_dir "results/"

Acknowledgement

RIC code makes use of ClearGrasp and Dalle-2 implementations. We sincerely thanks the authors for providing their implementations.

Citation

@misc{kasahara2023ric,
      title={RIC: Rotate-Inpaint-Complete for Generalizable Scene Reconstruction}, 
      author={Isaac Kasahara and Shubham Agrawal and Selim Engin and Nikhil Chavan-Dafle and Shuran Song and Volkan Isler},
      year={2023},
      eprint={2307.11932},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}