面向跨域行人再识别的虚拟数据生成与选择

doi:10.11996/JG.j.2095-302X.2023040775

图学学报 ›› 2023, Vol. 44 ›› Issue (4): 775-783.DOI: 10.11996/JG.j.2095-302X.2023040775

• 计算机图形学与虚拟现实 • 上一篇下一篇

面向跨域行人再识别的虚拟数据生成与选择

蔡益武¹(), 张雨佳¹, 张永飞¹^,²()

1.北京航空航天大学计算机学院北京市数字媒体实验室，北京 100191
2.北京航空航天大学虚拟现实技术与系统国家重点实验室，北京 100191

收稿日期:2022-11-30 接受日期:2022-12-26 出版日期:2023-08-31 发布日期:2023-08-16
通讯作者: 张永飞(1982-)，男，教授，博士。主要研究方向为计算机视觉等。E-mail：yfzhang@buaa.edu.cn
作者简介:
蔡益武(1999-)，男，硕士研究生。主要研究方向为行人再识别。E-mail：caiyiwu@buaa.edu.cn
基金资助:
国家自然科学基金项目(62072022)

Generation and selection of synthetic data for cross-domain person re-identification

CAI Yi-wu¹(), ZHANG Yu-jia¹, ZHANG Yong-fei¹^,²()

1. Beijing Key Laboratory of Digital Media, School of Computer Science and Engineering, Beihang University, Beijing 100191, China
2. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China

Received:2022-11-30 Accepted:2022-12-26 Online:2023-08-31 Published:2023-08-16
Contact: ZHANG Yong-fei (1982-), professor, Ph.D. His main research interests cover computer vision, etc. E-mail：yfzhang@buaa.edu.cn
About author:
CAI Yi-wu (1999-), master student. His main research interest covers person re-identification. E-mail：caiyiwu@buaa.edu.cn
Supported by:
National Natural Science Foundation of China(62072022)

摘要/Abstract

摘要：

针对当前基于深度学习的行人再识别模型依赖于大量标注数据的训练，其收集和标注代价极高；而现有的行人图像数据生成方法未考虑目标域数据特点，跨域性能有待提升的问题，提出一种面向跨域行人再识别的虚拟数据生成与选择算法。首先利用目标域前景信息如行人着装颜色分布指导虚拟3D人体模型生成，获得与真实人物整体着装较为相似的虚拟人物。接着引导模型专注于通过前景信息区分不同行人，在生成的虚拟数据上替换目标域背景信息，达到在像素级上提高源域数据质量的目的。最后，根据分布度量如Wasserstein Distance等度量源域和目标域的特征分布距离，在特征级上选择与目标域最接近的源域训练子集用以模型训练。实验结果表明，该方法优于现有的其他行人数据生成算法，可以显著提升行人再识别模型的跨域泛化性能。

关键词: 行人再识别, 数据生成, 虚拟引擎, 数据选择, 分布度量

Abstract:

The reliance of mainstream deep learning-based person re-identification models on large-scale labeled data for training is a costly process that requires extensive collection and labeling efforts. Additionally, the existing virtual data generation methods neglect to account for the characteristics of target domain, thereby compromising the performance of cross-domain re-identification. To address these issues, this paper proposed a synthetic data generation and selection algorithm for cross-domain person re-identification. First, this algorithm utilized the foreground information of the target domain, including the color distribution of individuals’ clothing, to guide the generation of virtual 3D human models. The background information of the target domain was employed to replace the background of source domain data. This served to enhance the data quality at the pixel level, while also guiding the model to distinguish different persons based on the foreground. Finally, the proposed method employed distribution metrics such as Wasserstein Distance to measure the feature distribution distance between the source domain and target domain. This distance was used to select the source domain training subset closest to the target domain for model training. The experimental results demonstrated the superiority of this method over other existing person virtual data generation algorithms, as it can significantly improve the cross-domain generalization performance of the person re-identification model.

Key words: person re-identification, data generation, virtual engine, data selection, distribution measure

中图分类号:

TP391

蔡益武, 张雨佳, 张永飞. 面向跨域行人再识别的虚拟数据生成与选择[J]. 图学学报, 2023, 44(4): 775-783.

CAI Yi-wu, ZHANG Yu-jia, ZHANG Yong-fei. Generation and selection of synthetic data for cross-domain person re-identification[J]. Journal of Graphics, 2023, 44(4): 775-783.

图/表 16

图1 现有虚拟数据集的行人图像

Fig. 1 Person Images from existing virtual datasets ((a) SyRI; (b) PersonX; (c) RandPerson; (d) UnrealPerson)

图2 Swin-Transformer模型结构[12]

Fig. 2 Swin-Transformer model structure[12]

图3 DukeMTMC-ReID数据集行人着装颜色分布

Fig. 3 Color distribution of person clothing in DukeMTMC-ReID

图4 基于前景信息的数据生成流程图

Fig. 4 Flow chart of data generation based on foreground

图5 按目标域着装分布指导生成的行人图像((a)目标域；(b)生成的虚拟行人)

Fig. 5 Person images generated according to the clothing distribution of target domain ((a) The target domain; (b) The generated virtual person)

图6 基于背景信息的样本生成流程图

Fig. 6 Flow chart of data generation based on background

图7 DukeMTMC-ReID中8个摄像机视频流的某一帧

Fig. 7 Frames of 8 camera video in DukeMTMC-ReID

图8 DukeMTMC-ReID对比图((a)纯背景；(b)高频背景)

Fig. 8 Comparison chart of DukeMTMC-ReID ((a) Pure background; (b) High frequency background)

图9 替换DukeMTMC-ReID背景前后的行人图像对比((a)替换背景前；(b)替换背景后)

Fig. 9 Person images before and after replacing the background of DukeMTMC-ReID ((a) Before replacing the background; (b) After replacing the background)

图10 IDM模型结构

Fig. 10 IDM model structure

图11 IDM模块详情

Fig. 11 IDM model detail

表1 训练集数据构成

Table 1 The composition of train set

数据集	行人数量 (人)	图像数量 (张)	摄像机数量 (台)
Market1501	751	12936	6
MSMT17	1041	32621	15
SyRI	100	56000	280
PersonX	410	88560	6
RandPerson	8000	132145	19
UnrealPerson	3000	120000	34
UnrealForDuke (Ours)	3000	120000	34

表2 测试集数据构成

Table 2 The composition of test set

数据集	行人数量 (人)	图像数量 (张)	摄像机数量 (台)
Duke query	702	2228	8
Duke gallery	1110	17661	8

表3 生成数据直接迁移的再识别效果

Table 3 Re-identification effects of direct transfering on real and virtual data

序号	训练数据集	Rank-1 (%)	mAP (%)
1	Market1501 (ICCV 2015)	56.7	36.5
2	MSMT17 (CVPR 2018)	67.1	46.8
3	SyRI (ECCV 2018)	38.9	18.2
4	PersonX (CVPR 2019)	49.4	28.9
5	RandPerson (MM 2020)	59.4	38.4
6	UnrealPerson (CVPR 2021)	69.7	49.4
7	UnrealForDuke (Ours)	71.6	51.0

表4 不同条件下生成数据直接迁移的再识别效果

Table 4 Re-identification effects of direct transfering on virtual data with different conditions

序号	随机着装	目标域行人着装颜色分布	目标域行人背景风格	Rank-1 (%)	mAP (%)
1	√	-	-	50.2	28.5
2	-	√	-	50.7	29.7
3	-	√	√	53.7	33.2

表5 不同条件下数据选择后直接迁移的再识别效果

Table 5 Re-identification effects of direct transfering on virtual data with different selection strategies

序号	挑选方式	Rank-1 (%)	mAP (%)
1	Baseline	80.8	70.9
2	RandByPic	80.5	71.1
3	RandByPid	81.2	71.6
4	KNN	81.3	71.7
5	FID	81.4	72.4
6	WD	82.0	71.5

参考文献 20

[1]	ZHENG L, YANG Y, HAUPTMANN A G. Person re-identification: past, present and future[EB/OL]. [2022-01-08]. https://arxiv.org/abs/1610.02984.
[2]	ZHANG X, LUO H, FAN X, et al. AlignedReID: surpassing human-level performance in person re-identification[EB/OL]. [2022-01-08]. https://arxiv.org/abs/1711.08184.
[3]	WEI L H, ZHANG S L, GAO W, et al. Person transfer GAN to bridge domain gap for person re-identification[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 79-88.
[4]	BĄK S, CARR P, LALONDE J F. Domain adaptation through synthesis for unsupervised person re-identification[C]// Computer Vision - ECCV 2018: 15th European Conference, Part XIII. New York: ACM, 2018: 193-209.
[5]	SUN X X, ZHENG L. Dissecting person re-identification from the viewpoint of viewpoint[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 608-617.
[6]	WANG Y N, LIAO S C, SHAO L. Surpassing real-world source training data: random 3D characters for generalizable person re-identification[C]// The 28th ACM International Conference on Multimedia. New York: ACM, 2020: 3422-3430.
[7]	ZHANG T Y, XIE L X, WEI L H, et al. UnrealPerson: an adaptive pipeline towards costless person re-identification[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 11501-11510.
[8]	CUI Y, SONG Y, SUN C, et al. Large scale fine-grained categorization and domain-specific transfer learning[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 4109-4118.
[9]	CHAKRABORTY S, UZKENT B, AYUSH K, et al. Efficient conditional pre-training for transfer learning[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2022: 4240-4249.
[10]	YAN X, ACUNA D, FIDLER S. Neural data server: a large-scale search engine for transfer learning data[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 3892-3901.
[11]	LUO H, WANG P C, XU Y, et al. Self-supervised pre-training for transformer-based person re-identification[EB/OL]. [2021-12-08]. https://arxiv.org/abs/2111.12084.
[12]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 9992-10002.
[13]	FU D P, CHEN D D, BAO J M, et al. Unsupervised pre-training for person re-identification[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 14745-14754.
[14]	DAI Y X, LIU J, SUN Y F, et al. IDM: an intermediate domain module for domain adaptive person re-ID[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 11844-11854.
[15]	HARTIGAN J A, WONG M A. Algorithm AS 136: a K-means clustering algorithm[J]. Applied Statistics, 1979, 28(1): 100. DOI URL
[16]	SCHUBERT E, SANDER J, ESTER M, et al. DBSCAN revisited, revisited[J]. ACM Transactions on Database Systems, 2017, 42(3): 1-21.
[17]	SHAMEEM M U S, FERDOUS R. An efficient k-means algorithm integrated with Jaccard distance measure for document clustering[C]// 2009 First Asian Himalayas International Conference on Internet. New York: IEEE Press, 2009: 1-6.
[18]	SHEN J, QU Y R, ZHANG W N, et al. Wasserstein distance guided representation learning for domain adaptation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 1.
[19]	ZHUANG Z J, WEI L H, XIE L X, et al. Rethinking the distribution gap of person re-identification with camera-based batch normalization[M]//Computer Vision - ECCV 2020. Cham: Springer International Publishing, 2020: 140-157.
[20]	SOLOVEITCHIK M, DISKIN T, MORIN E, et al. Conditional frechet inception distance[EB/OL]. [2021-11-21]. https://arxiv.org/abs/2103.11521.

面向跨域行人再识别的虚拟数据生成与选择

Generation and selection of synthetic data for cross-domain person re-identification

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 20

相关文章 2

编辑推荐

Metrics

本文评价

[1]	孙锐1,2，夏苗苗1,2，陆伟明1,2，张旭东1,2. 联合增强局部最大发生特征和k-KISSME 度量学习的行人再识别[J]. 图学学报, 2020, 41(3): 362-371.
[2]	胡彬 1,2，潘雨 1，丁卫平 1，邵叶秦 3，杨铖 1. 基于迁移学习的行人再识别[J]. 图学学报, 2018, 39(5): 886-891.