论文标题
PCSGAN：用于热图像转换的感知环合一的生成对抗网络
PCSGAN: Perceptual Cyclic-Synthesized Generative Adversarial Networks for Thermal and NIR to Visible Image Transformation
论文作者
论文摘要
在许多现实世界中，由于照明条件不佳，很难在可见光光谱（VIS）中捕获图像。但是，可以使用近红外（NIR）和热（THM）相机在这种情况下捕获图像。 NIR和THM图像包含有限的细节。因此，有必要将图像从THM/NIR转换为可以更好地理解。但是，由于较大的领域差异和缺乏丰富的数据集，这是非平凡的任务。如今，生成对抗网络（GAN）能够将图像从一个域转换为另一个域。大多数可用的基于GAN的方法都使用对抗性和像素损失（例如$ L_1 $或$ L_2 $）的组合作为培训的目标功能。在THM/NIR对VIS转换的情况下，转换图像的质量仍然无法使用此类目标函数达到标记。因此，需要更好的目标功能来改善转换图像的质量，细节和现实主义。引入了一个新的模型，用于查看图像转换，称为感知循环合成的生成对抗网络（PCSGAN）来解决这些问题。 PCSGAN使用感知（即基于特征）损失以及像素和对抗性损失的组合。定量和定性措施都用于判断PCSGAN模型在WHU-IIP面上的性能和RGB-NIR场景数据集。所提出的PCSGAN优于最先进的图像转换模型，包括Pix2Pix，Dualgan，Cyclean，PS2GAN和PAN，以及SSIM，MSE，PSNR和LPIPS评估指标。该代码可在https://github.com/kishankancharagunta/pcsgan上找到。
In many real world scenarios, it is difficult to capture the images in the visible light spectrum (VIS) due to bad lighting conditions. However, the images can be captured in such scenarios using Near-Infrared (NIR) and Thermal (THM) cameras. The NIR and THM images contain the limited details. Thus, there is a need to transform the images from THM/NIR to VIS for better understanding. However, it is non-trivial task due to the large domain discrepancies and lack of abundant datasets. Nowadays, Generative Adversarial Network (GAN) is able to transform the images from one domain to another domain. Most of the available GAN based methods use the combination of the adversarial and the pixel-wise losses (like $L_1$ or $L_2$) as the objective function for training. The quality of transformed images in case of THM/NIR to VIS transformation is still not up to the mark using such objective function. Thus, better objective functions are needed to improve the quality, fine details and realism of the transformed images. A new model for THM/NIR to VIS image transformation called Perceptual Cyclic-Synthesized Generative Adversarial Network (PCSGAN) is introduced to address these issues. The PCSGAN uses the combination of the perceptual (i.e., feature based) losses along with the pixel-wise and the adversarial losses. Both the quantitative and qualitative measures are used to judge the performance of the PCSGAN model over the WHU-IIP face and the RGB-NIR scene datasets. The proposed PCSGAN outperforms the state-of-the-art image transformation models, including Pix2pix, DualGAN, CycleGAN, PS2GAN, and PAN in terms of the SSIM, MSE, PSNR and LPIPS evaluation measures. The code is available at https://github.com/KishanKancharagunta/PCSGAN.
