Unsupervised Non-Rigid Image Distortion Removal via Grid Deformation

Paper | Video

Nianyi Li^3,1, Simron Thapa¹, Cameron Whyte², Albert Reed², Suren Jayasuriya², Jinwei Ye¹

¹Louisiana State University, ²Arizona State University, ³Clemson University

Accepted as poster in ICCV 2021

This repository constains a demo code for our Non-rigid image distortion removal network. Our implementation is in Pytorch.

Requirements

• Pytorch >=1.3
• Numpy
• PIL
• Opencv
• Jupyter
• Scikit-image
• Tqdm
• Matplotlib
• Torch-summary

Problem Definition

Many computer vision problems face difficulties when imaging through turbulent refractive media (e.g., air and water) due to the refraction and scattering of light. These effects cause geometric distortion that requires either handcrafted physical priors or supervised learning methods to remove.

Network Architecture

In this paper, we present a novel unsupervised network to recover the latent distortion-free image. The key idea is to model non-rigid distortions as deformable grids. Our network consists of a grid deformer that estimates the distortion field and an image generator that outputs the distortion-free image.

By leveraging the positional encoding operator, we can simplify the network structure while maintaining fine spatial details in the recovered images.

Results

Our method doesn't need to be trained on labeled data and has good transferability across various turbulent image datasets with different types of distortions. Extensive experiments on both simulated and real-captured turbulent images demonstrate that our method can remove both air and water distortions without much customization.

Citation

@InProceedings{Li_2021_ICCV,
    author    = {Li, Nianyi and Thapa, Simron and Whyte, Cameron and Reed, Albert W. and Jayasuriya, Suren and Ye, Jinwei},
    title     = {Unsupervised Non-Rigid Image Distortion Removal via Grid Deformation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {2522-2532}
}