IPCV's Tutored Reasearch and Development Project (TRDP): Image-to-Image Translation with Conditional Adversarial Networks
CycleGANs have become a very popular solution forthe image-to-image translation task because of their exceptionalresults and the capacity of avoiding the need of paired datasets.This quality eases the utilization of this model for any problemwhere the acquisition of a dataset is particularly difficult, forexample, in any medical imaging application. In our previouswork, we used two CycleGANs working together to create ageneralization of the model where the translation is performedbetween three domains instead of two. The addition of an extradomain proved to contribute to better results when the translationis performed between the other two domains. Nevertheless, thehigher complexity of the system that was presented makesthe training process unstable and the convergence tough toaccomplish. For all these factors, it is hard to analyse the resultsor to answer why and how the improvement occurs. Motivatedby this, in this work we propose to perform an analysis of theCycleGAN architecture, focusing on the visualization of the innerlayers of the generator. For this, we shift the application of themodel to a very basic colorization task using a simple toy datasetfor colorizing geometric shapes. To perform the experiments, wedeveloped an interface that allows to create a personalized toydataset and visualize the training and testing processes. Finally,we conducted a list of experiments were we try to shed light onthe behaviour of the CycleGAN filters and how the latent spaceof the U-Net encodes the information for the generation of thetranslated images.
The full report that describes the system can be downloaded here