Skip to content

Latest commit

 

History

History
95 lines (60 loc) · 2.01 KB

README.md

File metadata and controls

95 lines (60 loc) · 2.01 KB
Raw

Neural Texture

This repository implements Deferred Neural Rendering: Image Synthesis using Neural Textures .

Requirements

  • Python 3.6+
    • argparse
    • nni
    • NumPy
    • Pillow
    • pytorch
    • tensorboardX
    • torchvision
    • tqdm

File Organization

The root directory contains several subdirectories and files:

dataset/ --- custom PyTorch Dataset classes for loading included data
model/ --- custom PyTorch Module classes
util.py --- useful procedures
render.py --- render using texture and U-Net
render_texture.py --- render from RGB texture or neural texture
train.py --- optimize texture and U-Net jointly
train_texture.py --- optimize only texture
train_unet.py --- optimize U-Net using pretrained 3-channel texture

How to Use

Set up Environment

Install python >= 3.6 and create an environment.

Install requirements:

pip install -r requirements.txt

Prepare Data

We need 3 folders of data:

  • /data/frame/ with video frames .png files
  • /data/uv/ with uv-map .npy files, each shaped (H, W, 2)
  • /data/extrinsics/ with normalized camera extrinsics in .npy files, each shaped (3)

Each frame corresponds to one uv map and corresponding camera extrinsic parameters. They are named sequentially, from 0000 to xxxx .

We demonstrate 2 ways to prepare data. One way is to render training data, the code is at https://github.com/A-Dying-Pig/OpenGL_NeuralTexture. The other way is to reconstruct from real scene, the code is at https://github.com/gerwang/InfiniTAM .

Configuration

Rename config_example.py as config.py and set the parameters for training and rendering.

Train Jointly

python train.py [--args]

Train Texture

python train_texture.py [--args]

Train U-Net

python train_unet.py [--args]

Render by Texture

python render_texture.py [--args]

Render by Texture and U-Net Jointly

python render.py [--args]