Generation and selection of synthetic data for cross-domain person re-identification

doi:10.11996/JG.j.2095-302X.2023040775

Abstract

Abstract:

The reliance of mainstream deep learning-based person re-identification models on large-scale labeled data for training is a costly process that requires extensive collection and labeling efforts. Additionally, the existing virtual data generation methods neglect to account for the characteristics of target domain, thereby compromising the performance of cross-domain re-identification. To address these issues, this paper proposed a synthetic data generation and selection algorithm for cross-domain person re-identification. First, this algorithm utilized the foreground information of the target domain, including the color distribution of individuals’ clothing, to guide the generation of virtual 3D human models. The background information of the target domain was employed to replace the background of source domain data. This served to enhance the data quality at the pixel level, while also guiding the model to distinguish different persons based on the foreground. Finally, the proposed method employed distribution metrics such as Wasserstein Distance to measure the feature distribution distance between the source domain and target domain. This distance was used to select the source domain training subset closest to the target domain for model training. The experimental results demonstrated the superiority of this method over other existing person virtual data generation algorithms, as it can significantly improve the cross-domain generalization performance of the person re-identification model.

Key words: person re-identification, data generation, virtual engine, data selection, distribution measure

CLC Number:

TP391

CAI Yi-wu, ZHANG Yu-jia, ZHANG Yong-fei. Generation and selection of synthetic data for cross-domain person re-identification[J]. Journal of Graphics, 2023, 44(4): 775-783.

Figures/Tables 16

Fig. 1 Person Images from existing virtual datasets ((a) SyRI; (b) PersonX; (c) RandPerson; (d) UnrealPerson)

Fig. 2 Swin-Transformer model structure[12]

Fig. 3 Color distribution of person clothing in DukeMTMC-ReID

Fig. 4 Flow chart of data generation based on foreground

Fig. 5 Person images generated according to the clothing distribution of target domain ((a) The target domain; (b) The generated virtual person)

Fig. 6 Flow chart of data generation based on background

Fig. 7 Frames of 8 camera video in DukeMTMC-ReID

Fig. 8 Comparison chart of DukeMTMC-ReID ((a) Pure background; (b) High frequency background)

Fig. 9 Person images before and after replacing the background of DukeMTMC-ReID ((a) Before replacing the background; (b) After replacing the background)

Fig. 10 IDM model structure

Fig. 11 IDM model detail

Table 1 The composition of train set

数据集	行人数量 (人)	图像数量 (张)	摄像机数量 (台)
Market1501	751	12936	6
MSMT17	1041	32621	15
SyRI	100	56000	280
PersonX	410	88560	6
RandPerson	8000	132145	19
UnrealPerson	3000	120000	34
UnrealForDuke (Ours)	3000	120000	34

Table 2 The composition of test set

数据集	行人数量 (人)	图像数量 (张)	摄像机数量 (台)
Duke query	702	2228	8
Duke gallery	1110	17661	8

Table 3 Re-identification effects of direct transfering on real and virtual data

序号	训练数据集	Rank-1 (%)	mAP (%)
1	Market1501 (ICCV 2015)	56.7	36.5
2	MSMT17 (CVPR 2018)	67.1	46.8
3	SyRI (ECCV 2018)	38.9	18.2
4	PersonX (CVPR 2019)	49.4	28.9
5	RandPerson (MM 2020)	59.4	38.4
6	UnrealPerson (CVPR 2021)	69.7	49.4
7	UnrealForDuke (Ours)	71.6	51.0

Table 4 Re-identification effects of direct transfering on virtual data with different conditions

序号	随机着装	目标域行人着装颜色分布	目标域行人背景风格	Rank-1 (%)	mAP (%)
1	√	-	-	50.2	28.5
2	-	√	-	50.7	29.7
3	-	√	√	53.7	33.2

Table 5 Re-identification effects of direct transfering on virtual data with different selection strategies

序号	挑选方式	Rank-1 (%)	mAP (%)
1	Baseline	80.8	70.9
2	RandByPic	80.5	71.1
3	RandByPid	81.2	71.6
4	KNN	81.3	71.7
5	FID	81.4	72.4
6	WD	82.0	71.5

References 20

[1]	ZHENG L, YANG Y, HAUPTMANN A G. Person re-identification: past, present and future[EB/OL]. [2022-01-08]. https://arxiv.org/abs/1610.02984.
[2]	ZHANG X, LUO H, FAN X, et al. AlignedReID: surpassing human-level performance in person re-identification[EB/OL]. [2022-01-08]. https://arxiv.org/abs/1711.08184.
[3]	WEI L H, ZHANG S L, GAO W, et al. Person transfer GAN to bridge domain gap for person re-identification[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 79-88.
[4]	BĄK S, CARR P, LALONDE J F. Domain adaptation through synthesis for unsupervised person re-identification[C]// Computer Vision - ECCV 2018: 15th European Conference, Part XIII. New York: ACM, 2018: 193-209.
[5]	SUN X X, ZHENG L. Dissecting person re-identification from the viewpoint of viewpoint[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 608-617.
[6]	WANG Y N, LIAO S C, SHAO L. Surpassing real-world source training data: random 3D characters for generalizable person re-identification[C]// The 28th ACM International Conference on Multimedia. New York: ACM, 2020: 3422-3430.
[7]	ZHANG T Y, XIE L X, WEI L H, et al. UnrealPerson: an adaptive pipeline towards costless person re-identification[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 11501-11510.
[8]	CUI Y, SONG Y, SUN C, et al. Large scale fine-grained categorization and domain-specific transfer learning[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 4109-4118.
[9]	CHAKRABORTY S, UZKENT B, AYUSH K, et al. Efficient conditional pre-training for transfer learning[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2022: 4240-4249.
[10]	YAN X, ACUNA D, FIDLER S. Neural data server: a large-scale search engine for transfer learning data[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 3892-3901.
[11]	LUO H, WANG P C, XU Y, et al. Self-supervised pre-training for transformer-based person re-identification[EB/OL]. [2021-12-08]. https://arxiv.org/abs/2111.12084.
[12]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 9992-10002.
[13]	FU D P, CHEN D D, BAO J M, et al. Unsupervised pre-training for person re-identification[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 14745-14754.
[14]	DAI Y X, LIU J, SUN Y F, et al. IDM: an intermediate domain module for domain adaptive person re-ID[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 11844-11854.
[15]	HARTIGAN J A, WONG M A. Algorithm AS 136: a K-means clustering algorithm[J]. Applied Statistics, 1979, 28(1): 100. DOI URL
[16]	SCHUBERT E, SANDER J, ESTER M, et al. DBSCAN revisited, revisited[J]. ACM Transactions on Database Systems, 2017, 42(3): 1-21.
[17]	SHAMEEM M U S, FERDOUS R. An efficient k-means algorithm integrated with Jaccard distance measure for document clustering[C]// 2009 First Asian Himalayas International Conference on Internet. New York: IEEE Press, 2009: 1-6.
[18]	SHEN J, QU Y R, ZHANG W N, et al. Wasserstein distance guided representation learning for domain adaptation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 1.
[19]	ZHUANG Z J, WEI L H, XIE L X, et al. Rethinking the distribution gap of person re-identification with camera-based batch normalization[M]//Computer Vision - ECCV 2020. Cham: Springer International Publishing, 2020: 140-157.
[20]	SOLOVEITCHIK M, DISKIN T, MORIN E, et al. Conditional frechet inception distance[EB/OL]. [2021-11-21]. https://arxiv.org/abs/2103.11521.