GP-GAN: Towards Realistic High-Resolution Image Blending (ACMMM 2019, oral)

[Project] [Paper] [Demo] [Related Work: A2RL (for Auto Image Cropping)] [Colab]
Official Chainer implementation of GP-GAN: Towards Realistic High-Resolution Image Blending

Overview

source	destination	mask	composited	blended

The author's implementation of GP-GAN, the high-resolution image blending algorithm described in:
"GP-GAN: Towards Realistic High-Resolution Image Blending"
Huikai Wu, Shuai Zheng, Junge Zhang, Kaiqi Huang

Given a mask, our algorithm can blend the source image and the destination image, generating a high-resolution and realsitic blended image. Our algorithm is based on deep generative models Wasserstein GAN.

Contact: Hui-Kai Wu (huikaiwu@icloud.com)

Citation

@article{wu2017gp,
  title   = {GP-GAN: Towards Realistic High-Resolution Image Blending},
  author  = {Wu, Huikai and Zheng, Shuai and Zhang, Junge and Huang, Kaiqi},
  journal = {ACMMM},
  year    = {2019}
}

Getting started

• The code is tested with python==3.5 and chainer==6.3.0 on Ubuntu 16.04 LTS.

• Download the code from GitHub:

  git clone https://github.com/wuhuikai/GP-GAN.git
  cd GP-GAN

• Install the requirements:

  pip install -r requirements/test/requirements.txt

• Download the pretrained model blending_gan.npz or unsupervised_blending_gan.npz from Google Drive, and then put them in the folder models.

• Run the script for blending_gan.npz:

  python run_gp_gan.py --src_image images/test_images/src.jpg --dst_image images/test_images/dst.jpg --mask_image images/test_images/mask.png --blended_image images/test_images/result.png

  Or run the script for unsupervised_blending_gan.npz:

  python run_gp_gan.py --src_image images/test_images/src.jpg --dst_image images/test_images/dst.jpg --mask_image images/test_images/mask.png --blended_image images/test_images/result.png --supervised False

• Type python run_gp_gan.py --help for a complete list of the arguments.

Train GP-GAN step by step

Train Blending GAN

• Download Transient Attributes Dataset here.

• Crop the images in each subfolder:

  python crop_aligned_images.py --data_root [Path for imageAlignedLD in Transient Attributes Dataset]

• Train Blending GAN:

  python train_blending_gan.py --data_root [Path for cropped aligned images of Transient Attributes Dataset]

• Training Curve

  

• Visual Result

  Training Set	Validation Set

Training Unsupervised Blending GAN

• Requirements

  pip install git+git://github.com/mila-udem/fuel.git@stable

• Download the hdf5 dataset of outdoor natural images: ourdoor_64.hdf5 (1.4G), which contains 150K landscape images from MIT Places dataset.

• Train unsupervised Blending GAN:

  python train_wasserstein_gan.py --data_root [Path for outdoor_64.hdf5]

• Training Curve

• Samples after training

  

Visual results

Mask	Copy-and-Paste	Modified-Poisson	Multi-splines	Supervised GP-GAN	Unsupervised GP-GAN