PCSGAN: Perceptual Cyclic-Synthesized Generative Adversarial Networks
  for Thermal and NIR to Visible Image Transformation
  In many real world scenarios, it is difficult to capture the images in the
visible light spectrum (VIS) due to bad lighting conditions. However, the
images can be captured in such scenarios using Near-Infrared (NIR) and Thermal
(THM) cameras. The NIR and THM images contain the limited details. Thus, there
is a need to transform the images from THM/NIR to VIS for better understanding.
However, it is non-trivial task due to the large domain discrepancies and lack
of abundant datasets. Nowadays, Generative Adversarial Network (GAN) is able to
transform the images from one domain to another domain. Most of the available
GAN based methods use the combination of the adversarial and the pixel-wise
losses (like $L_1$ or $L_2$) as the objective function for training. The
quality of transformed images in case of THM/NIR to VIS transformation is still
not up to the mark using such objective function. Thus, better objective
functions are needed to improve the quality, fine details and realism of the
transformed images. A new model for THM/NIR to VIS image transformation called
Perceptual Cyclic-Synthesized Generative Adversarial Network (PCSGAN) is
introduced to address these issues. The PCSGAN uses the combination of the
perceptual (i.e., feature based) losses along with the pixel-wise and the
adversarial losses. Both the quantitative and qualitative measures are used to
judge the performance of the PCSGAN model over the WHU-IIP face and the RGB-NIR
scene datasets. The proposed PCSGAN outperforms the state-of-the-art image
transformation models, including Pix2pix, DualGAN, CycleGAN, PS2GAN, and PAN in
terms of the SSIM, MSE, PSNR and LPIPS evaluation measures. The code is
available at https://github.com/KishanKancharagunta/PCSGAN.
