Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)

Official Tensorflow implementation of Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)
Jeong-gi Kwak, Youngsaeng Jin, Yuanming Li, Dongsik Yoon, Donghyeon Kim and Hanseok Ko
British Machine Vision Conference (BMVC), 2021

Intro

Night → Day (BDD100K)

Rainy night → Day (Alderdey)

Architecture

Our generator has asymmetric structure for editing day→night and night→day. Please refer our paper for details

Envs

git clone https://github.com/jgkwak95/AU-GAN.git
cd AU-GAN

# Create virtual environment
conda create -y --name augan python=3.6.7
conda activate augan

conda install tensorflow-gpu==1.14.0   # Tensorflow 1.14
pip install --no-cache-dir -r requirements.txt

Preparing datasets

Night → Day
Berkeley DeepDrive dataset contains 100,000 high resolution images of the urban roads for autonomous driving.

Rainy night → Day
Alderley dataset consists of images of two domains, rainy night and daytime. It was collected while driving the same route in each weather environment.

Please download datasets and then construct them following ForkGAN

Pretrained Model

Download the pretrained model for BDD100K(256x512) here and unzip it to ./check/bdd_exp/bdd100k_256/

Training

# Alderley (256x512)
python main_uncer.py --dataset_dir alderley
                     --phase train
                     --experiment_name alderley_exp
                     --batch_size 8 
                     --load_size 286 
                     --fine_size 256 
                     --use_uncertainty True

# BDD100k (256x512)
python main_uncer.py --dataset_dir bdd100k 
                     --phase train
                     --experiment_name bdd_exp
                     --batch_size 8 
                     --load_size 286
                     --fine_size 256 
                     --use_uncertainty True

Test

# Alderley (256x512)
python main_uncer.py --dataset_dir alderley
                     --phase test
                     --experiment_name alderley_exp
                     --batch_size 1 
                     --load_size 286 
                     --fine_size 256 
                    

# BDD100k (256x512)
python main_uncer.py --dataset_dir bdd100k
                     --phase test
                     --experiment_name bdd_exp
                     --batch_size 1 
                     --load_size 286 
                     --fine_size 256 
                    

Additional results

More results in paper and supplementary

Uncertainty map

Citation

If our code is helpful your research, please cite our paper:

@article{kwak2021adverse,
  title={Adverse weather image translation with asymmetric and uncertainty-aware GAN},
  author={Kwak, Jeong-gi and Jin, Youngsaeng and Li, Yuanming and Yoon, Dongsik and Kim, Donghyeon and Ko, Hanseok},
  journal={arXiv preprint arXiv:2112.04283},
  year={2021}
}

Acknowledgments

Our code is bulided upon the ForkGAN implementation.