基于残差增强注意力的跨模态行人重识别

doi:10.11996/JG.j.2095-302X.2023010033

摘要/Abstract

摘要：

跨模态行人重识别主要面临 2 个问题：①成像机制不同所导致的红外图像和可见光图像之间的模态差异；②图像特征的身份判别性不足导致的类内差异。针对上述 2 个问题，基于残差增强注意力的跨模态行人重识别方法被提出用来提高行人特征的模态不变性和身份判别性。首先，设计网络浅层参数独立、网络深层参数共享的双路卷积神经网络作为骨干网络。然后，分析现有注意力机制存在的全局弱化，设计了残差增强注意力方法解决该问题，提升注意力机制的性能，将其分别应用在网络浅层的通道维度和深层的空间位置上，提升模型对于模态差异的消除能力和行人特征的身份鉴别能力。在 SYSU-MM01 和 RegDB 2 个数据集上进行的实验证明了该方法的先进性，大量的对比实验也充分证明本文方法的有效性。

关键词:

"> 行人重识别, 残差增强, 注意力机制, 模态不变性, 神经网络

Abstract:

Cross modality person re-identification mainly faces two problems: ① Modality discrepancies between infrared and visible images caused by different imaging mechanisms. ② Intra-class discrepancies caused by the insufficient identity discrimination of features. In order to address the above two problems, a cross modality person re-identification method based on residual enhanced attention was proposed to improve the modality invariance and identity discrimination of pedestrian features. First, with non-shared parameters at the shallow network and shared parameters at the deep layer a dual-stream convolutional neural network was designed as the backbone. Then, the problem of global weakening in the existing attention mechanism was analyzed, and a residual enhancement method was designed to solve this problem and improve the performance of the attention mechanism. It was applied to the shallow channel dimension and deep spatial location of the network respectively. Sufficient experiments on the two datasets SYSU-MM01 and RegDB have proved the effectiveness of the method.

Key words:

"> person re-identification, residual enhancement, attention mechanism, modality invariance, neural network

中图分类号:

TP 391

邵文斌, 刘玉杰, 孙晓瑞, 李宗民. 基于残差增强注意力的跨模态行人重识别[J]. 图学学报, 2023, 44(1): 33-40.

SHAO Wen-bin, LIU Yu-jie, SUN Xiao-rui, LI Zong-min . Cross modality person re-identification based on residual enhanced attention[J]. Journal of Graphics, 2023, 44(1): 33-40.

参考文献

[1] YE M, SHEN J, LIN G, et al. Deep learning for person re-identification: a survey and outlook[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 2872-2893.

[2] 杨文娟, 王文明, 王全玉, 等. 基于感知哈希和视觉词袋模型的图像检索方法[J]. 图学学报, 2019, 40(3): 519-524. YANG W J, WANG W M, WANG Q Y, et al. Image retrieval method based on perceptual hash algorithm and bag of visual words[J]. Journal of Graphics, 2019, 40(3): 519-524 (in Chinese).

[3] ZHAO H, TIAN M, SUN S, et al. Spindle net: person re-identification with human body region guided feature decomposition and fusion[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 1077-1085.

[4] GE Y X, ZHU F, CHEN D P, et al. Self-paced contrastive learning with hybrid memory for domain adaptive object re-id[EB/OL]. [2021-12-08]. https://blog.csdn.net/NGUever15/ article/details/120556059.

[5] CHEN H, WANG Y, LAGADEC B. Joint generative and contrastive learning for unsupervised person re-identification[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 2004-2013.

[6] WU A, ZHENG W, YU H, et al. RGB-infrared cross-modality person re-identification[C]//2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 5380-5389.

[7] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// The IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7132-7141.

[8] WOO S, PARK J, LEE J Y, et al. Cbam: convolutional block attention module[C]// 2018 European Conference on Computer Vision. Cham: Springer International Publishin, 2018: 3-19.

[9] DAI P Y, JI R R, WANG H, WU Q, el al. Cross-modality person re-identification with generative adversarial training[EB/OL]. [2021-12-08]. https://www.ijcai.org/Proceedings/ 2018/0094.pdf.

[10] WANG Z X, WANG Z, ZHENG Y. Learning to reduce dual-level discrepancy for infrared-visible person re-identification[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 618-626.

[11] WANG G, ZHANG T, CHENG J, et al. RGB-infrared cross-modality person re-identification via joint pixel and feature alignment[C]//2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 3623-3632.

[12] ZHAO Z, LIU B, CHU Q, et al. Joint color-irrelevant consistency learning and identity-aware modality adaptation for visible-infrared cross modality person re-identification[C]// The AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2021: 3520-3528.

[13] YE M, LAN X, WANG Z, et al. Bi-directional centerconstrained top-ranking for visible thermal person re-identification[C]//IEEE Transactions on Information Pernsics and Security. New York: IEEE Press, 2020: 407-419.

[14] HAO Y, WANG N, LI J, el al. Hsme: hypersphere manifold embedding for visible thermal person re-identification[C]//The AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2019: 8385-8392.

[15] JIA M X, ZHAI Y P, LU S J, et al. A similarity inference metric for RGB-infrared cross-modality person re-identification[EB/OL]. [2021-12-08]. https://arxiv.org/abs/ 2007.01504.

[16] LI D, WEI X, HONG X, el al. Infrared-visible cross-modal person re-identification with an x modality[C]//The AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 4610-4617.

[17] YE M, SHEN J, CRANDALL D, et al. Dynamic dual-attentive aggregation learning for visible-infrared person re-identification[C]//2020 European Conference on Computer Vision – ECCV 2020. Cham: Springer International Publishin, 2020: 229-247.

[18] CHEN Y, WAN L, LI Z, et al. Neural feature search for rgb-infrared person re-identification[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New York: IEEE Press, 2021: 587-597.

[19] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[EB/OL]. [2021-12-08]. https://blog.csdn. net/toda666/article/details/80384915.

[20] LUO H, GU Y, LIAO X, et al. Bag of tricks and a strong baseline for deep person re-identification[EB/OL]. [2021-12-08]. https://openaccess.thecvf.com/content_CVPRW_2019/papers/ TRMTMCT/Luo_Bag_of_Tricks_and_a_Strong_Baseline_for_ Deep_Person_CVPRW_2019_paper.pdf.

[21] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2005: 886-893.

[22] NGUYEN D T, HONG H G, KIM K W, et al. Person recognition system based on a combination of body images from visible light and thermal cameras[J]. Sensors: Basel, Switzerland, 2017, 17(3): E605.

[23] WANG G, ZHANG T, YANG Y, et al. Cross-modality paired-images generation for RGB-infrared person re-identification[C]//The AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 12144-12151.

[24] DENG J, DONG W, SOCHER R, et al. Imagenet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2009: 248-255.

[25] KINGMA D, BA J. Adam: a method for stochastic optimization[EB/OL]. [2021-12-08]. https://arxiv.org/abs/1412. 6980.

[26] YE M, LAN X, LENG Q, et al. Cross-modality person re-identification via modality-aware collaborative ensemble learning[C]//2020 IEEE Transactions on Image Processing. New York: IEEE Press, 2020: 9387-9399.

[27] LU Y, WU Y, LIU B, et al. Cross-modality person re-identification with shared-specific feature transfer[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 13376-13386.

[28] YE M, RUAN W J, DU B, et al. Channel augmented joint learning for visible-infrared recognition[C]//2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 13567-13576.

[29] SELVARAJU R, COGSWELL M, DAS A. Grad-cam: visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2): 336-359.