U-GAN Model for Infrared and Visible Images Fusion

红外与可见光图像融合的U-GAN模型

¹ School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, China
² School of Artificial Intelligence, Changchun University of Science and Technology, Changchun 130022, China

Received: 8 October 2019

Abstract

Infrared and visible image fusion is an effective method to solve the lack of single sensor imaging. The purpose is that the fusion images are suitable for human eyes and conducive to the next application and processing. In order to solve the problems of incomplete feature extraction, loss of details, and less samples of common data sets, it is not conducive to training, an end-to-end network architecture for image fusion is proposed. U-net is introduced into image fusion, and the final fusion result is obtained by using the generative adversarial network. Through its special convolution structure, the important feature information is extracted to the maximum extent, and the sample does not need to be cut to avoid the problem of reducing the fusion accuracy, but also to improve the training speed. Then the U-net extracted feature is confronted with the discriminator containing infrared image, and the generator model is obtained. The experimental results show that the present algorithm can obtain the fusion image with clear outline, prominent texture and obvious target. SD, SF, SSIM, AG and other indicators are obviously improved.

摘要

红外与可见光图像进行融合是解决单一传感器成像不足的有效手段，目的是得到适合人眼并有利于下一步应用和处理的融合图像。为解决大部分方法特征提取不全面，细节纹理丢失及公共数据集样本较少不利于训练等问题，提出一种用于图像融合的端到端网络结构。将U-net特有的卷积结构用于图像融合，最大程度地提取并保留源图像的重要特征信息。再通过生成对抗网络得到最后的融合结果，将U-net提取的特征输入生成器与包含红外图像的鉴别器进行对抗，得到训练模型。实验结果表示，所提算法能够得到轮廓清晰、纹理突出、目标明显的融合图像，SD、SF、SSIM、AG等指标明显得到提升。