Infrared and visible image fusion algorithm based on split-attention residual networks

基于拆分注意力残差网络的红外和可见光图像融合算法

¹ Coastal Defense Academy, Naval Aeronautical University, Yantai 264000, China
² No. 32127 Unit of PLA, Dalian 116100, China

Received: 14 March 2022

Abstract

在红外和可见光图像融合算法中, 图像信息的丢失始终是制约融合图像质量提升的关键问题, 为此, 提出了一种基于拆分注意力残差网络的红外和可见光图像融合算法, 使用带有拆分注意力模块的深层残差网络拓展感受野和提高跨通道信息融合能力, 运用平滑最大值单元函数作为激活函数进一步提升网络性能; 特征提取后运用零相位分量分析和归一化算法得到融合权重后完成图像融合。实验结果表明, 融合后的图像细节丰富, 边缘锐利; 在峰值信噪比、结构相似性指数度量和基于梯度的融合性能等指标上与经典的6种算法相比均有不同程度提升。

摘要

In the infrared and visible image fusion algorithm, the loss of image information is always the key problem to restrict the improvement of fusion image quality. Therefore, an infrared and visible image fusion algorithm based on split-attention residual network is proposed. The residual network with split-attention block is used to expand the receptive field and improve the ability of cross-channel information fusion, and the smooth maximum unit function is used as the activation function to further improve the network performance. Then, the extracted feature map is whitened by using the zero-phase component analysis method to project it in the same subspace, and the initial weight map is obtained by L₁-norm. Then the bicubic interpolation algorithm is used for upsampling and the softmax function is used for weight normalization to obtain the weight matrix consistent with the size of the original image. Finally, the weighted average strategy is used to weighted average the original infrared and visible images to obtain the final fused image. In order to verify the performance of the algorithm, the subjective and objective evaluation is compared with the six classical fusion algorithms in experiment. In the subjective evaluation, the fusion image of the present algorithm is in detail, which not only reflects the thermal information in the infrared image, but also retains the texture details of the visible image, with sharp edges, high color restoration and natural appearance. Among the three evaluation indexes of peak signal-to-noise ratio, structural similarity index measure and gradient-based fusion performance, the present algorithm improves at least 1.78%, 2.00% and 3.10% by comparing with the other six algorithms.