Consistent and unbiased teacher model research for domain adaptive object detection

doi:10.11996/JG.j.2095-302X.2025010114

Abstract

Abstract:

As a significant approach, the self-training method has significantly enhanced the performance of domain adaptive object detection methods. The self-training method primarily predicts target domain data through a teacher network, and then selects high-confidence predictions as pseudo-labels to guide student network training. However, due to significant domain differences between the source and target domains, the pseudo-labels generated by the teacher network are often of poor quality, adversely impacting student network training and reducing the performance of object detection. To address this challenge, a consistent and unbiased teacher (CUT) model for domain adaptive object detection was proposed. Firstly, an adaptive threshold generation (ATG) module was designed within the teacher network. The ATG module utilized a Gaussian mixture model (GMM) during training to generate adaptive thresholds for each image, ensuring temporal consistency of pseudo-label quantities and thereby enhancing their quality. Secondly, a prediction-guided sample selection (PSS) strategy was introduced, which leveraged predictions from the region proposal network within the teacher network to guide the selection of positive and negative samples for the student network. The PSS strategy effectively aligned the selected samples with real outcomes, thereby mitigating the impact of low-quality pseudo-labels on the student network. Furthermore, to improve detection performance for small objects and challenging objects with fewer instances, a mixed domain augmentation (MDA) module was devised to generate mixed-domain images containing random information from both the source and target-like domains to supervise student network training. Extensive experiments conducted on three scenario datasets demonstrated the effectiveness of the proposed CUT, with performance improvements of 4.0%, 5.8%, and 3.7%, respectively. Notably, the proposed CUT model applied the self-training method for the first time to address the problem of large domain disparities between visual images and infrared images.

Key words: domain adaption, object detection, self-training, pseudo-labels, consistency

CLC Number:

CHENG Xudong, SHI Caijuan, GAO Weixiang, WANG Sen, DUAN Changyu, YAN Xiaodong. Consistent and unbiased teacher model research for domain adaptive object detection[J]. Journal of Graphics, 2025, 46(1): 114-125.

Figures/Tables 18

References 43

[1]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[2]	TIAN Z, SHEN C H, CHEN H, et al. FCOS: a simple and strong anchor-free object detector[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(4): 1922-1933.
[3]	CHEN Y H, LI W, SAKARIDIS C, et al. Domain adaptive faster R-CNN for object detection in the wild[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 3339-3348.
[4]	CAI Q, PAN Y W, NGO C W, et al. Exploring object relation in mean teacher for cross-domain detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 11449-11458.
[5]	SONG G L, LIU Y, WANG X G. Revisiting the sibling head in object detector[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 11560-11569.
[6]	WANG X J, YANG X Y, ZHANG S L, et al. Consistent-teacher: towards reducing inconsistent pseudo-targets in semi- supervised object detection[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 3240-3249.
[7]	KENNERLEY M, WANG J G, VEERAVALLI B, et al. CAT: exploiting inter-class dynamics for domain adaptive object detection[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 16541-16550.
[8]	王兵, 徐裴, 张兴鹏. 傅里叶增强的无偏跨域目标检测研究[J]. 计算机科学与探索, 2024, 18(9): 2436-2448. DOI
	WANG B, XU P, ZAHNG X P. Research on Fourier augmented unbiased cross-domain object detection[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(9): 2436-2448 (in Chinese).
[9]	李晶晶, 孟利超, 张可, 等. 领域自适应研究综述[J]. 计算机工程, 2021, 47(6): 1-13. DOI
	LI J J, MENG L C, ZHANG K, et al. Review of studies on domain adaptation[J]. Computer Engineering, 2021, 47(6): 1-13 (in Chinese). DOI
[10]	彭国琴, 张浩, 徐丹. 基于域自适应的云南重彩画无监督情感识别[J]. 图学学报, 2022, 43(4): 641-650.
	PENG G Q, ZHANG H, XU D. Unsupervised emotion recognition of Yunnan Heavy Color Paintings based on domain adaptation[J]. Journal of Graphics, 2022, 43(4): 641-650 (in Chinese).
[11]	YU Y, ZHAI Y C, ZHANG Y. Align and Adapt: a two-stage adaptation framework for unsupervised domain adaptation[C]// The 30th ACM International Conference on Multimedia. New York: ACM, 2022: 4723-4732.
[12]	张桂梅, 陶辉, 鲁飞飞, 等. 基于双源判别器的域自适应城市场景语义分割[J]. 图学学报, 2023, 44(5): 907-917. DOI
	ZHANG G M, TAO H, LU F F, et al. Domain adaptive urban scene semantic segmentation based on dual-source discriminator[J]. Journal of Graphics, 2023, 44(5): 907-917 (in Chinese).
[13]	ZHAO L, WANG L M. Task-specific inconsistency alignment for domain adaptive object detection[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 14197-14206.
[14]	SAITO K, USHIKU Y, HARADA T, et al. Strong-weak distribution alignment for adaptive object detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 6949-6958.
[15]	ZHU X G, PANG J M, YANG C Y, et al. Adapting object detectors via selective cross-domain alignment[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 687-696.
[16]	HSU C C, TSAI Y H, LIN Y Y, et al. Every pixel matters: center-aware feature alignment for domain adaptive object detector[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 733-748.
[17]	WANG H A, JIA S L, ZENG T Y, et al. Triple feature disentanglement for one-stage adaptive object detection[C]// The 38th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2024: 5401-5409.
[18]	FU H L, LIU H H, YUAN J, et al. YOLO-Adaptor: a fast adaptive one-stage detector for non-aligned visible-infrared Object Detection[EB/OL]. (2024-04-25) [2024-05-29]. https://ieeexplore.ieee.org/abstract/document/10508329.
[19]	SHI C J, ZHENG Y F, CHEN Z. Domain adaptive thermal object detection with unbiased granularity alignment[J]. ACM Transactions on Multimedia Computing, Communications and Applications, 2024, 20(9): 274.
[20]	LI W Y, LIU X Y, YUAN Y X. SIGMA: semantic-complete graph matching for domain adaptive object detection[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5281-5290.
[21]	TARVAINEN A, VALPOLA H. Mean teachers are better role models: weight-averaged consistency targets improve semi- supervised deep learning results[C]// The 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 1195-1204.
[22]	DENG J H, LI W, CHEN Y H, et al. Unbiased mean teacher for cross-domain object detection[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 4089-4099.
[23]	LI Y J, DAI X L, MA C Y, et al. Cross-domain adaptive teacher for object detection[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 7571-7580.
[24]	CHEN M L, CHEN W J, YANG S C, et al. Learning domain adaptive object detection with probabilistic teacher[EB/OL]. [2024-05-01]. https://proceedings.mlr.press/v162/chen22b.html.
[25]	DENG J H, XU D L, LI W, et al. Harmonious teacher for cross-domain object detection[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 23829-23838.
[26]	CAO S C, JOSHI D, GUI L Y, et al. Contrastive mean teacher for domain adaptive object detectors[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 23839-23848.
[27]	GE Z, LIU S T, WANG F, et al. YOLOX: exceeding YOLO series in 2021[EB/OL]. [2024-05-29]. https://xueshu.baidu.com/usercenter/paper/show?paperid=195c06c0472x0xe01m0006h0j7739694.
[28]	CORDTS M, OMRAN M, RAMOS S, et al. The cityscapes dataset for semantic urban scene understanding[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 3213-3223.
[29]	SAKARIDIS C, DAI D X, VAN GOOL L. Semantic foggy scene understanding with synthetic data[J]. International Journal of Computer Vision, 2018, 126(9): 973-992.
[30]	MARNISSI M A, FRADI H, SAHBANI A, et al. Feature distribution alignments for object detection in the thermal domain[J]. The Visual Computer, 2023, 39(3): 1081-1093.
[31]	JOHNSON-ROBERSON M, BARTO C, MEHTA R, et al. Driving in the matrix: can virtual worlds replace human-generated annotations for real world tasks?[C]// 2017 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2017: 746-753.
[32]	CHEN C Q, ZHENG Z B, DING X H, et al. Harmonizing transferability and discriminability for adapting object detectors[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 8866-8875.
[33]	VS V, GUPTA V, OZA P, et al. MeGA-CDA: memory guided attention for category-aware unsupervised domain adaptive object detection[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 4514-4524.
[34]	LI W Y, LIU X Y, YAO X W, et al. SCAN: cross domain object detection with semantic conditioned adaptation[C]// The 36th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2022: 1421-1428.
[35]	PIAO Z Q, TANG L B, ZHAO B J. Unsupervised domain-adaptive object detection via localization regression alignment[EB/OL]. (2023-06-19) [2024-05-29]. https://ieeexplore.ieee.org/document/10154573.
[36]	LI J L, XU R S, MA J, et al. Domain adaptive object detection for autonomous driving under foggy weather[C]// 2023 IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2023: 612-622.
[37]	LIU Y B, WANG J H, HUANG C, et al. CIGAR: cross-modality graph reasoning for domain adaptive object detection[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 23776-23786.
[38]	PANG J, LIU W F, ZHANG B F, et al. MCNet: magnitude consistency network for domain adaptive object detection under inclement environments[J]. Pattern Recognition, 2024, 145: 109947.
[39]	GUO Y S, YU H, XIE S R, et al. DSCA: a dual semantic correlation alignment method for domain adaptation object detection[J]. Pattern Recognition, 2024, 150: 110329.
[40]	LIU Y B, WANG J H, WANG W J, et al. CRADA: cross domain object detection with cyclic reconstruction and decoupling adaptation[J]. IEEE Transactions on Multimedia, 2024, 26: 6250-6261.
[41]	XU C D, ZHAO X R, JIN X, et al. Exploring categorical regularization for domain adaptive object detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 11721-11730.
[42]	WU A M, LIU R, HAN Y H, et al. Vector-decomposed disentanglement for domain-invariant object detection[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 9322-9331.
[43]	LI X F, CHEN W J, XIE D, et al. A free lunch for unsupervised domain adaptive object detection without source data[C]// The 35th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2021: 8474-8481.

模型	模型来源	Person	Rider	Car	Truck	Bus	Train	Mcycle	Bicycle	mAP
EPM^[16]	ECCV'20	41.9	38.7	56.7	22.6	41.5	26.8	24.6	35.5	36.0
HTCN^[32]	CVPR'20	33.2	47.5	47.9	31.6	47.4	40.9	32.3	37.1	39.8
MeGA-CDA^[33]	CVPR'21	37.7	49.0	52.4	25.4	49.2	46.9	34.5	39.0	41.8
TIA^[13]	CVPR'22	34.8	46.3	49.7	31.1	52.1	48.6	37.7	38.1	42.3
SCAN^[34]	AAAI'22	41.7	43.9	57.3	28.7	48.6	48.7	31.0	37.3	42.1
SIGMA^[20]	CVPR'22	44.0	43.9	60.3	31.6	50.4	51.5	31.7	40.6	44.2
PT^[24]	ICML'22	40.2	48.8	59.7	30.7	51.8	30.6	35.4	44.5	42.7
LRA^[35]	TNNLS'23	45.6	47.1	59.7	32.1	52.4	44.6	34.8	39.9	44.5
DA-Detect^[36]	WACV'23	36.5	46.7	54.3	30.3	51.2	48.7	31.6	39.1	42.3
PT+CMT^[26]	CVPR'23	42.3	51.7	64.0	26.0	42.7	37.1	42.5	44.0	43.8
CIGAR^[37]	CVPR'23	45.3	45.3	61.6	32.1	50.0	51.0	31.9	40.4	44.7
MCNet^[38]	PR'24	48.0	50.4	62.4	26.5	46.3	34.3	37.1	43.1	43.5
DSCA^[39]	PR'24	40.5	44.3	60.2	29.3	50.1	49.5	30.6	37.6	42.7
CRADA^[40]	TMM'24	47.0	48.3	64.1	28.7	52.3	38.6	35.1	41.5	44.5
UGA^[19]	TOMM'24	39.5	51.1	57.5	35.0	51.4	44.2	37.3	43.6	45.0
Baseline^[22]	CVPR'21	33.0	46.7	48.6	34.1	56.5	46.8	30.4	37.3	41.7
CUT(本文)	-	41.1	52.0	57.3	33.6	52.2	45.7	39.4	44.0	45.7

模型	模型来源	Person	Rider	Car	Truck	Bus	Train	Mcycle	Bicycle	mAP
EPM^[16]	ECCV'20	41.9	38.7	56.7	22.6	41.5	26.8	24.6	35.5	36.0
HTCN^[32]	CVPR'20	33.2	47.5	47.9	31.6	47.4	40.9	32.3	37.1	39.8
MeGA-CDA^[33]	CVPR'21	37.7	49.0	52.4	25.4	49.2	46.9	34.5	39.0	41.8
TIA^[13]	CVPR'22	34.8	46.3	49.7	31.1	52.1	48.6	37.7	38.1	42.3
SCAN^[34]	AAAI'22	41.7	43.9	57.3	28.7	48.6	48.7	31.0	37.3	42.1
SIGMA^[20]	CVPR'22	44.0	43.9	60.3	31.6	50.4	51.5	31.7	40.6	44.2
PT^[24]	ICML'22	40.2	48.8	59.7	30.7	51.8	30.6	35.4	44.5	42.7
LRA^[35]	TNNLS'23	45.6	47.1	59.7	32.1	52.4	44.6	34.8	39.9	44.5
DA-Detect^[36]	WACV'23	36.5	46.7	54.3	30.3	51.2	48.7	31.6	39.1	42.3
PT+CMT^[26]	CVPR'23	42.3	51.7	64.0	26.0	42.7	37.1	42.5	44.0	43.8
CIGAR^[37]	CVPR'23	45.3	45.3	61.6	32.1	50.0	51.0	31.9	40.4	44.7
MCNet^[38]	PR'24	48.0	50.4	62.4	26.5	46.3	34.3	37.1	43.1	43.5
DSCA^[39]	PR'24	40.5	44.3	60.2	29.3	50.1	49.5	30.6	37.6	42.7
CRADA^[40]	TMM'24	47.0	48.3	64.1	28.7	52.3	38.6	35.1	41.5	44.5
UGA^[19]	TOMM'24	39.5	51.1	57.5	35.0	51.4	44.2	37.3	43.6	45.0
Baseline^[22]	CVPR'21	33.0	46.7	48.6	34.1	56.5	46.8	30.4	37.3	41.7
CUT(本文)	-	41.1	52.0	57.3	33.6	52.2	45.7	39.4	44.0	45.7

模型	模型来源	Car	Bicycle	Person	mAP
SWDA^[14]	CVPR'19	58.9	32.0	32.3	41.4
CR-DA-DET^[41]	CVPR'20	58.3	37.6	46.3	47.4
HTCN^[32]	CVPR'20	56.3	37.9	33.1	42.4
EPM^[16]	ECCV'20	53.8	39.0	41.0	44.6
VDD^[42]	ICCV'21	66.6	49.2	45.8	53.9
TIA^[13]	CVPR'22	67.8	44.8	48.8	53.8
UDAT^[30]	TVC'23	66.8	49.3	43.4	53.2
UGA^[19]	TOMM'24	68.5	48.0	51.6	56.0
Baseline^[22]	CVPR'21	65.3	43.5	49.5	52.8
CUT(本文)	-	68.3	52.7	54.9	58.6

模型	模型来源	Car	Bicycle	Person	mAP
SWDA^[14]	CVPR'19	58.9	32.0	32.3	41.4
CR-DA-DET^[41]	CVPR'20	58.3	37.6	46.3	47.4
HTCN^[32]	CVPR'20	56.3	37.9	33.1	42.4
EPM^[16]	ECCV'20	53.8	39.0	41.0	44.6
VDD^[42]	ICCV'21	66.6	49.2	45.8	53.9
TIA^[13]	CVPR'22	67.8	44.8	48.8	53.8
UDAT^[30]	TVC'23	66.8	49.3	43.4	53.2
UGA^[19]	TOMM'24	68.5	48.0	51.6	56.0
Baseline^[22]	CVPR'21	65.3	43.5	49.5	52.8
CUT(本文)	-	68.3	52.7	54.9	58.6

模型	模型来源	AP on Car/%
SCDA^[15]	CVPR'19	43.0
EPM^[16]	ECCV'20	43.2
HTCN^[32]	CVPR'20	42.5
SED^[43]	AAAI'21	42.5
MeGA-CDA^[33]	CVPR'21	44.8
SCAN^[34]	AAAI'22	45.8
SIGMA^[20]	CVPR'22	45.8
Baseline^[22]	CVPR'21	43.1
CUT(本文)	-	46.8