基于周期一致性和动态记忆增强的无监督无人机目标跟踪

doi:10.11996/JG.j.2095-302X.2025061281

图学学报 ›› 2025, Vol. 46 ›› Issue (6): 1281-1291.DOI: 10.11996/JG.j.2095-302X.2025061281

• 图像处理与计算机视觉 • 上一篇下一篇

基于周期一致性和动态记忆增强的无监督无人机目标跟踪

肖凯¹^,²(), 袁玲¹^,², 储珺¹^,²()

¹ 南昌航空大学图像处理与模式识别江西省重点实验室，江西南昌 330063
² 南昌航空大学软件学院，江西南昌 330063

收稿日期:2025-03-06 接受日期:2025-05-07 出版日期:2025-12-30 发布日期:2025-12-27
通讯作者:储珺(1967-)，女，教授，博士。主要研究方向为计算机视觉、图像处理和深度学习。E-mail：chuj@nchu.edu.cn
第一作者:肖凯(1999-)，男，硕士研究生。主要研究方向为目标跟踪和无监督学习。E-mail：xiaok9900@163.com
基金资助:
江西省研究生创新专项(YC2023-S747)

Unsupervised cycle-consistent learning with dynamic memory-augmented for unmanned aerial vehicle videos tracking

XIAO Kai¹^,²(), YUAN Ling¹^,², CHU Jun¹^,²()

¹ Jiangxi Provincial Key Laboratory of Image Processing and Pattern Recognition, Nanchang Hangkong University, Nanchang Jiangxi 330063, China
² School of Software Engineering, Nanchang Hangkong University, Nanchang Jiangxi 330063, China

Received:2025-03-06 Accepted:2025-05-07 Published:2025-12-30 Online:2025-12-27
First author：XIAO Kai (1999-), master student. His main research interests cover object tracking and unsupervised learning. E-mail：xiaok9900@163.com
Supported by:
Innovative Special Fund for Graduate Students in Jiangxi Province(YC2023-S747)

摘要/Abstract

摘要：

针对无人机(UAV)视频数据集采集成本高，现有数据普遍存在规模有限、应用场景单一，且现有无监督目标跟踪方法通常只用于通用数据集设计，对UAV的复杂场景难以学习可靠信息等问题，提出一种无监督UAV目标跟踪模型，其基于时间周期一致性与动态记忆增强。首先，将显著性目标检测引入无标签的对象发现，并与无监督光流技术结合，引入基于图像熵的动态规划，提高伪标签的质量。其次，为视频中的每一帧定义权重，并利用这些权重进行单帧训练，以更全面地利用每一帧的信息。最后，借鉴长短期记忆网络的思想，将记忆队列转变为动态记忆队列。设计自注意力分支且作为记忆队列的门控机制，并控制队列的记忆与遗忘，在不增加队列长度的同时，实现长跨度下的目标特征变化学习。该方法在UAV数据集上的准确率达到了68%，领先于其他无监督跟踪器，与一般有监督跟踪器的性能持平。在一般场景数据集上也与其他无监督跟踪器性能近似，准确率达到77%。在UAV数据集和一般场景数据集上的实验结果表明，其在快速运动和大尺度变化场景性能方面有较好提高。

关键词: 目标跟踪, 无人机, 无监督学习, 注意力机制, 孪生网络

Abstract:

The collection of UAV (unmanned aerial vehicle) video datasets is costly and faces issues such as limited quantity, low quality, and scenario constraints. To address these challenges, an unsupervised UAV-object-tracking model based on temporal cycle consistency and dynamic memory enhancement was proposed. First, salient-object detection was introduced for unlabeled object discovery. By combining salient object detection with unsupervised optical flow techniques and incorporating dynamic programming based on image entropy, the quality of pseudo-labels was improved. Second, a weight is defined for each frame in the video, and these weights are utilized for single-frame training to fully leverage the information from all frames. Finally, inspired by long short-term memory (LSTM) networks, the memory queue was transformed into a dynamic memory queue, along with a self-attention branch designed to control its updates. Target-features changes over long spans were learned without increasing the queue length. The proposed method achieved 68% accuracy on UAV datasets, outperforming other unsupervised trackers and matching typical supervised-tracker performance. On general scene datasets, it attained 77?% accuracy, comparable to other unsupervised trackers. Experimental results on both UAV and general scene datasets demonstrated that the proposed method achieved excellent performance in scenarios involving rapid motion and large-scale variations.

Key words: object tracking, unmanned aerial vehicle, unsupervised learning, attention mechanism, twin network

中图分类号:

肖凯, 袁玲, 储珺. 基于周期一致性和动态记忆增强的无监督无人机目标跟踪[J]. 图学学报, 2025, 46(6): 1281-1291.

XIAO Kai, YUAN Ling, CHU Jun. Unsupervised cycle-consistent learning with dynamic memory-augmented for unmanned aerial vehicle videos tracking[J]. Journal of Graphics, 2025, 46(6): 1281-1291.

图/表 16

参考文献 26

[1]	王三泉, 王璐, 储珺, 等. 基于未知类别少样本学习的开放世界目标定位[J]. 南昌航空大学学报(自然科学版), 2024, 38(3): 33-44, 71.
	WANG S Q, WANG L, CHU J, et al. Open world object localization based on few shot learning of unknown categories[J]. Journal of Nanchang Hangkong University (Natural Sciences), 2024, 38(3): 33-44, 71 (in Chinese).
[2]	程旭东, 史彩娟, 高炜翔, 等. 面向域自适应目标检测的一致无偏教师模型[J]. 图学学报, 2025, 46(1): 114-125. DOI
	CHENG X D, SHI C J, GAO W X, et al. Consistent and unbiased teacher model research for domain adaptive object detection[J]. Journal of Graphics, 2025, 46(1): 114-125 (in Chinese). DOI
[3]	蔡嘉诚, 董方敏, 孙水发, 等. 无监督单目深度估计研究综述[J]. 计算机科学, 2024, 51(2): 117-134. DOI
	CAI J C, DONG F M, SUN S F, et al. Unsupervised learning of monocular depth estimation: a survey[J]. Computer Science, 2024, 51(2): 117-134 (in Chinese).
[4]	王文鹏, 秦寅畅, 师文轩. 工业缺陷检测无监督深度学习方法综述[J]. 计算机应用, 2025, 45(5): 1658-1670. DOI
	WANG W P, QIN Y C, SHI W X. Review of unsupervised deep learning methods for industrial defect detection[J]. Journal of Computer Applications, 2025, 45(5): 1658-1670 (in Chinese). DOI
[5]	DWIBEDI D, AYTAR Y, TOMPSON J, et al. Temporal cycle-consistency learning[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 1801-1810.
[6]	冯尊登, 王洪元, 林龙, 等. 基于多分支注意网络与相似度学习策略的无监督行人重识别[J]. 图学学报, 2023, 44(2): 280-290. DOI
	FENG Z D, WANG H Y, LIN L, et al. Unsupervised person re-identification with multi-branch attention network and similarity learning strategy[J]. Journal of Graphics, 2023, 44(2): 280-290 (in Chinese).
[7]	WANG X L, JABRI A, EFROS A A. Learning correspondence from the cycle-consistency of time[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 2561-2571.
[8]	WANG N, SONG Y B, MA C, et al. Unsupervised deep tracking[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 1308-1317.
[9]	WANG N, ZHOU W G, SONG Y B, et al. Unsupervised deep representation learning for real-time tracking[J]. International Journal of Computer Vision, 2021, 129(2): 400-418. DOI
[10]	ZHENG J L, MA C, PENG H W, et al. Learning to track objects from unlabeled videos[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 13526-13535.
[11]	SHEN Q H, QIAO L, GUO J Y, et al. Unsupervised learning of accurate siamese tracking[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 8091-8100.
[12]	SHI X J, CHEN Z R, WANG H, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting[C]// The 29th International Conference on Neural Information Processing Systems - Volume 1. New York:ACM, 2015: 802-810.
[13]	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]// 2016 Computer Vision-ECCV 2016 Workshops. Cham: Springer, 2016: 850-865.
[14]	LI B, YAN J J, WU W, et al. High performance visual tracking with Siamese region proposal network[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 8971-8980.
[15]	LI B, WU W, WANG Q, et al. SiamRPN++: evolution of Siamese visual tracking with very deep networks[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 4277-4286.
[16]	郑晗, 储珺. 目标检测中的特征融合方法[J]. 南昌航空大学学报(自然科学版), 2022, 36(4): 59-67.
	ZHENG H, CHU J. Feature fusion method for object detection[J]. Journal of Nanchang Hangkong University (Natural Sciences), 2022, 36(4): 59-67 (in Chinese).
[17]	CHEN Z D, ZHONG B N, LI G R, et al. Siamese box adaptive network for visual tracking[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 6667-6676.
[18]	GUO D Y, WANG J, CUI Y, et al. SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 6268-6276.
[19]	储珺, 林文杰, 徐鹏. 目标检测中特征不匹配问题研究进展[J]. 南昌航空大学学报(自然科学版), 2021, 35(3): 1-8.
	CHU J, LIN W J, XU P. Reviews of feature mismatch in object detection[J]. Journal of Nanchang Hangkong University (Natural Sciences), 2021, 35(3): 1-8 (in Chinese).
[20]	FU C H, CAO Z A, LI Y M, et al. Siamese anchor proposal network for high-speed aerial tracking[C]// 2021 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2021: 510-516.
[21]	CAO Z A, FU C H, YE J J, et al. SiamAPN++: Siamese attentional aggregation network for real-time UAV tracking[C]// 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE Press, 2021: 3086-3092.
[22]	XING D T, EVANGELIOU N, TSOUKALAS A, et al. Siamese transformer pyramid networks for real-time UAV tracking[C]// 2022 IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2022: 1898-1907.
[23]	SIO C H, MA Y J, SHUAI H H, et al. S2SiamFC: self-supervised fully convolutional Siamese network for visual tracking[C]// The 28th ACM International Conference on Multimedia. New York: ACM, 2020: 1948-1957.
[24]	YUAN W H, WANG M Y, CHEN Q F. Self-supervised object tracking with cycle-consistent Siamese networks[C]// 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York: IEEE Press, 2020: 10351-10358.
[25]	WU Q Q, WAN J, CHAN A B. Progressive unsupervised learning for visual object tracking[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 2992-3001.
[26]	LIU L, ZHANG J N, HE R F, et al. Learning by analogy: reliable supervision from transformations for unsupervised optical flow estimation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 6488-6497.

方法	光流方法	显著性检测	图像熵DP	IOU得分
方法	光流方法	显著性检测	图像熵DP	UAV123	UAV20L
Baseline	√			0.284	0.277
Ours	√	√		0.391	0.330
Ours		√	√	0.401	0.393
Ours	√		√	0.368	0.408
Ours	√	√	√	0.547	0.503

方法	光流方法	显著性检测	图像熵DP	IOU得分
方法	光流方法	显著性检测	图像熵DP	UAV123	UAV20L
Baseline	√			0.284	0.277
Ours	√	√		0.391	0.330
Ours		√	√	0.401	0.393
Ours	√		√	0.368	0.408
Ours	√	√	√	0.547	0.503

跟踪器	视频利用率η
USOT^[10]	36.7
UTC-DMT (Ours)	72.9

跟踪器	视频利用率η
USOT^[10]	36.7
UTC-DMT (Ours)	72.9

跟踪器	帧跨度S
LUDT	10.0
USOT^[10]	41.1
UTC-DMT (Ours)	68.5

基于周期一致性和动态记忆增强的无监督无人机目标跟踪

Unsupervised cycle-consistent learning with dynamic memory-augmented for unmanned aerial vehicle videos tracking

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 26

相关文章 15

编辑推荐

Metrics

本文评价

跟踪器(帧长度设置)	P
USOT^[10](7)	0.668
UTC-DMT(5)	0.671
UTC-DMT(6)	0.677
UTC-DMT(7)	0.681
UTC-DMT(8)	0.681

算法	类别	评价指标
算法	类别	AUC	P
SiamFC^[13]	有监督	0.498	0.726
SiamRPN^[14]	有监督	0.527	0.748
SiamRPN++^[15]	有监督	0.613	0.807
SiamAPN^[17]	有监督	0.566	0.752
SiamTPN^[22]	有监督	0.660	0.858
UDT^[8]	无监督	0.480	0.672
USOT^[10]	无监督	0.502	0.668
Ours	无监督	0.514	0.681

算法	类别	评价指标
算法	类别	Suc	Pre	NPre	FPS
SiamFC^[13]	有监督	0.571	0.533	0.663	55
SiamRPN^[14]	有监督	-	-	-	-
SiamRPN++^[15]	有监督	0.733	0.694	0.800	38
UDT^[8]	无监督	0.563	0.495	0.633	63
USOT^[10]	无监督	0.599	0.551	0.682	50
ULAST^[11]	无监督	0.649	0.585	0.725	40
Ours	无监督	0.614	0.570	0.691	52

[1]	薄文, 琚晨, 刘维青, 张焱, 胡晶晶, 程婧晗, 张常有. 基于退化感知时序建模的装备维保时机预测方法[J]. 图学学报, 2025, 46(6): 1233-1246.
[2]	于男男, 孟政宇, 房友江, 孙传昱, 殷雪峰, 张强, 魏小鹏, 杨鑫. 融合多频超图的图像语义分割方法[J]. 图学学报, 2025, 46(6): 1267-1273.
[3]	张馨匀, 张力文, 周李, 罗笑南. 基于图像分块交互的咖啡果实成熟度预测模型[J]. 图学学报, 2025, 46(6): 1274-1280.
[4]	曹璐静, 鲁鹏. 一种基于多参考图的视频上色方法[J]. 图学学报, 2025, 46(6): 1316-1326.
[5]	刘伯凯, 殷雪峰, 孙传昱, 葛慧林, 魏子麒, 姜雨彤, 朴海音, 周东生, 杨鑫. 基于深度强化学习的无人机三维场景导航方法研究[J]. 图学学报, 2025, 46(5): 1010-1017.
[6]	左屿琪, 张云峰, 张秋悦, 徐英城. 基于超图表示学习和Transformer模型优化的知识感知推荐[J]. 图学学报, 2025, 46(5): 1050-1060.
[7]	翟永杰, 翟邦朝, 胡哲东, 杨珂, 王乾铭, 赵晓瑜. 基于自适应特征融合金字塔与注意力机制的输电线路绝缘子缺陷检测方法[J]. 图学学报, 2025, 46(5): 950-959.
[8]	杨佳熙, 于乐天, 包骐瑞, 毕胜, 麻晓斗, 杨晟琦, 姜雨彤, 方建儒, 魏小鹏, 杨鑫. 面向高光子通量环境的目标深度估计方法[J]. 图学学报, 2025, 46(4): 756-762.
[9]	牛杭, 葛鑫雨, 赵晓瑜, 杨珂, 王乾铭, 翟永杰. 基于改进YOLOv8的防振锤缺陷目标检测算法[J]. 图学学报, 2025, 46(3): 532-541.
[10]	于冰, 程广, 黄东晋, 丁友东. 基于双流网络融合的三维人体网格重建[J]. 图学学报, 2025, 46(3): 625-634.
[11]	胡悦, 孙智达, 黄惠. 面向无人机路径规划的可视分析系统[J]. 图学学报, 2025, 46(3): 655-665.
[12]	雷玉林, 刘利刚. 基于深度强化学习的可缓冲的物体运输和装箱[J]. 图学学报, 2025, 46(3): 697-708.
[13]	张立立, 杨康, 张珂, 魏薇, 李晶, 谭洪鑫, 张翔宇. 面向柴油车辆排放黑烟的改进型YOLOv8检测算法研究[J]. 图学学报, 2025, 46(2): 249-258.
[14]	崔克彬, 耿佳昌. 基于EE-YOLOv8s的多场景火灾迹象检测算法[J]. 图学学报, 2025, 46(1): 13-27.
[15]	陈冠豪, 徐丹, 贺康建, 施洪贞, 张浩. 基于转置注意力和CNN的图像超分辨率重建网络[J]. 图学学报, 2025, 46(1): 35-46.

算法	类别	评价指标
算法	类别	EAO	Acc	Rob
SiamFC^[13]	有监督	0.188	0.503	0.585
SiamRPN^[14]	有监督	-	-	-
SiamRPN++^[15]	有监督	0.414	0.600	0.234
UDT^[8]	无监督	0.230	0.490	0.412
USOT^[10]	无监督	0.290	0.564	0.435
ULAST^[11]	无监督	0.347	0.569	0.304
Ours	无监督	0.304	0.571	0.422

算法	类别	评价指标
算法	类别	EAO	Acc	Rob
SiamFC^[13]	有监督	0.188	0.503	0.585
SiamRPN^[14]	有监督	-	-	-
SiamRPN++^[15]	有监督	0.414	0.600	0.234
UDT^[8]	无监督	0.230	0.490	0.412
USOT^[10]	无监督	0.290	0.564	0.435
ULAST^[11]	无监督	0.347	0.569	0.304
Ours	无监督	0.304	0.571	0.422