DistgDisp: Disentangling Mechanism for Light Field Disparity Estimation

This is the PyTorch implementation of the disparity estimation method in our paper "Disentangling Light Fields for Super-Resolution and Disparity Estimation". Please refer to our paper and project page for details.

Preparation:

Requirement:

• PyTorch 1.3.0, torchvision 0.4.1. The code is tested with python=3.6, cuda=9.0.
• A single GPU with cuda memory larger than 12 GB is required to reproduce the inference time reported in our paper.

Datasets:

• We used the HCI 4D LF benchmark for training and evaluation. Please refer to the benchmark website for details.

Path structure:

├──./datasets/
│    ├── training
│    │    ├── antinous
│    │    │    ├── gt_disp_lowres.pfm
│    │    │    ├── valid_mask.png
│    │    │    ├── input_Cam000.png
│    │    │    ├── input_Cam001.png
│    │    │    ├── ...
│    │    ├── boardgames
│    │    ├── ...
│    ├── validation
│    │    ├── backgammon
│    │    │    ├── gt_disp_lowres.pfm
│    │    │    ├── input_Cam000.png
│    │    │    ├── input_Cam001.png  
│    │    │    ├── ...
│    │    ├── boxes
│    |    ├── ...
│    ├── test
│    │    ├── bedroom
│    │    │    ├── input_Cam000.png
│    │    │    ├── input_Cam001.png  
│    │    │    ├── ...
│    │    ├── bicycle
│    |    ├── herbs
│    |    ├── origami

Train:

• Set the hyper-parameters in parse_args() if needed. We have provided our default settings in the realeased codes.
• Run train.py to perform network training.
• Checkpoint will be saved to ./log/.

Test on your own LFs:

• Place the input LFs into ./demo_input (see the attached examples).
• Run test.py to perform inference on each test scene.
• The result files (i.e., scene_name.pfm) will be saved to ./Results/.

Reproduce the scores on the HCI 4D LF benchmark:

• Perform inference on each scene separately to generate .pfm result files.
• Download groundtruth disparity images (i.e., gt_disp_lowres.pfm) and use the official evaluation toolkit to obtain quantitative results.

Reproduce the inference time reported in our paper:

• Run test_inference_time.py to reproduce the inference time reported in our paper. Note that, the inference need to be performed on a GPU with a cuda memory larger than 12 GB.

Results:

Quantitative Results:

Visual Comparisons:

Screenshot on the HCI 4D LF Benchmark (July 2020):

Performance on real LFs and extended applications:

Citiation

If you find this work helpful, please consider citing:

@Article{DistgLF,
    author    = {Wang, Yingqian and Wang, Longguang and Wu, Gaochang and Yang, Jungang and An, Wei and Yu, Jingyi and Guo, Yulan},
    title     = {Disentangling Light Fields for Super-Resolution and Disparity Estimation},
    journal   = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)}, 
    year      = {2022},   
}

Contact

Welcome to raise issues or email to wangyingqian16@nudt.edu.cn for any question regarding this work.