基于深度迁移学习的图像隐私目标检测研究

doi:10.11996/JG.j.2095-302X.2023061112

摘要/Abstract

摘要：

针对当前社交媒体平台缺乏图像隐私泄露预警机制的问题，提出基于YOLOv8模型的图像隐私目标检测优化方案，以降低用户分享图像时泄露隐私的风险。以YOLOv8作为基线模型，将瓶颈转换器(BoT)模块融入主干网络，以捕获全局上下文信息，建模长距离依赖关系。同时引入加权双向特征金字塔网络(BIFPN)结构改进颈部网络，促进多尺度特征的深度融合。在此基础上，基于深度迁移学习方法，对YOLOv8预训练模型进行微调并训练，以实现图像隐私的自动化检测。通过LabelImg标注工具构建隐私图像数据集，在迁移学习方式下，将常见的YOLO系列模型与改进的YOLOv8相比较。结果表明：YOLOv8在基线模型中的表现较好，而本文改进模型的F1值达到0.885，mAP@.5值达到0.927，相较于YOLOv8分别提升了4.0%和3.4%，其检测精度效果显著，能够应对图像隐私检测的应用场景。

关键词: 图像隐私, 目标检测, YOLOv8, 全局上下文信息, 迁移学习

Abstract:

Addressing the absence of an early warning mechanism for image privacy leakage in current social media platforms, an optimization scheme for image privacy target detection based on the YOLOv8 model was proposed, thus reducing the risk of privacy leakage when users share images. Building upon the YOLOv8 baseline model, the bottleneck transformer (BoT) module was integrated into the backbone network, capturing global contextual information and modeling long-range dependencies of targets. Concurrently, the bidirectional feature pyramid network (BiFPN) structure was introduced to improve the neck network, facilitating the deep fusion of multi-scale features. On this basis, based on the deep transfer learning method, the YOLOv8 pre-training model was fine-tuned and trained to achieve automatic detection of image privacy. A privacy image dataset was constructed using the LabelImg annotation tool, and the common YOLO series model was compared with the improved YOLOv8 in the transfer learning mode. The results demonstrated that YOLOv8 exhibited strong performance in the baseline model, while the F1 and mAP@.5 values of the improved model proposed in this study reached 0.885 and 0.927, respectively, reflecting a 4.0% and 3.4% increase compared with YOLOv8. This significantly enhanced detection accuracy was well-suited for image privacy detection in various application scenarios.

Key words: image privacy, target detection, YOLOv8, global contextual information, transfer learning

中图分类号:

TP391.4

王大阜, 王静, 石宇凯, 邓志文, 贾志勇. 基于深度迁移学习的图像隐私目标检测研究[J]. 图学学报, 2023, 44(6): 1112-1120.

WANG Da-fu, WANG Jing, SHI Yu-kai, DENG Zhi-wen, JIA Zhi-yong. Research on image privacy detection based on deep transfer learning[J]. Journal of Graphics, 2023, 44(6): 1112-1120.

图/表 12

参考文献 23

[1]	李凤华, 孙哲, 牛犇, 等. 跨社交网络的隐私图片分享框架[J]. 通信学报, 2019, 40(7): 1-13. DOI
	LI F H, SUN Z, NIU B, et al. Privacy-preserving photo sharing framework cross different social network[J]. Journal on Communications, 2019, 40(7): 1-13 (in Chinese). DOI
[2]	赵景欣, 岳星辉, 冯崇朋, 等. 基于通用数据保护条例的数据隐私安全综述[J]. 计算机研究与发展, 2022, 59(10): 2130-2163.
	ZHAO J X, YUE X H, FENG C P, et al. Survey of data privacy security based on general data protection regulation[J]. Journal of Computer Research and Development, 2022, 59(10): 2130-2163 (in Chinese).
[3]	王树义, 刘赛, 马峥. 基于深度迁移学习的微博图像隐私分类研究[J]. 数据分析与知识发现, 2020, 4(10): 80-92.
	WANG S Y, LIU S, MA Z. Microblog image privacy classification with deep transfer learning[J]. Data Analysis and Knowledge Discovery, 2020, 4(10): 80-92 (in Chinese).
[4]	ZERR S, SIERSDORFER S, HARE J, et al. Privacy-aware image classification and search[C]// The 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2012: 35-44.
[5]	TONGE A, CARAGEA C. Privacy prediction of images shared on social media sites using deep features[EB/OL]. [2023-02-12]. https://arxiv.org/abs/1510.08583.pdf.
[6]	TONGE A, CARAGEA C. Image privacy prediction using deep neural networks[J]. ACM Transactions on the Web, 2020, 14(2): Article No. 7: 1-7: 32.
[7]	郑剑, 冷碧玉. 多类别图像数据分类隐私保护算法[J]. 科学技术与工程, 2020, 20(29): 12007-12013.
	ZHENG J, LENG B Y. Privacy protection algorithm of multi-category image data classification[J]. Science Technology and Engineering, 2020, 20(29): 12007-12013 (in Chinese).
[8]	李小波, 李阳贵, 郭宁, 等. 融合注意力机制的YOLOv5口罩检测算法[J]. 图学学报, 2023, 44(1): 16-25. DOI
	LI X B, LI Y G, GUO N, et al. Mask detection algorithm based on YOLOv5 integrating attention mechanism[J]. Journal of Graphics, 2023, 44(1): 16-25 (in Chinese).
[9]	蒋心璐, 陈天恩, 王聪, 等. 大田环境下的农业害虫图像小目标检测算法[J/OL]. 计算机工程. https://doi.org/10.19678/j.issn.1000-3428.0067030.
	JIANG X L, CHEN T E, WANG C, et al. Small object detection algorithm of agricultural pest image in field environment[J/OL]. Computer Engineering. https://doi.org/10.19678/j.issn.1000-3428.0067030 (in Chinese).
[10]	夏烨, 雷哓晖, 祁雁楠, 等. 基于改进Ghost-YOLOv5s- BiFPN算法检测梨树花序[J]. 智慧农业: 中英文, 2022, 4(3): 108-119.
	XIA Y, LEI X H, QI Y N, et al. Detection of pear inflorescence based on improved ghost-YOLOv5s-BiFPN algorithm[J]. Smart Agriculture: English and Chinese, 2022, 4(3): 108-119 (in Chinese).
[11]	余平平, 林耀海, 赖云锋, 等. 融合BiFPN和YOLOv5s的密集型原木端面检测方法[J]. 林业工程学报, 2023, 8(1): 126-134.
	YU P P, LIN Y H, LAI Y F, et al. Dense log end face detection method using the hybrid of BiFPN and YOLOv5s[J]. Journal of Forestry Engineering, 2023, 8(1): 126-134 (in Chinese).
[12]	杨观赐, 杨静, 苏志东, 等. 改进的YOLO特征提取算法及其在服务机器人隐私情境检测中的应用[J]. 自动化学报, 2018, 44(12): 2238-2249.
	YANG G C, YANG J, SU Z D, et al. An improved YOLO feature extraction algorithm and its application to privacy situation detection of social robots[J]. Acta Automatica Sinica, 2018, 44(12): 2238-2249 (in Chinese).
[13]	毛雨晴, 赵奎. 基于改进YOLOv5的多任务安全人头检测算法[J]. 计算机工程, 2022, 48(8): 136-143. DOI
	MAO Y Q, ZHAO K. Multi-task secure head-detection algorithm based on improved YOLOv5[J]. Computer Engineering, 2022, 48(8): 136-143 (in Chinese). DOI
[14]	WU C L, YANG C K, LIN Y L. Privacy protection and beautification of cornea images[J]. Multimedia Tools and Applications, 2022, 81(22): 32421-32448. DOI
[15]	NEPAL U, ESLAMIAT H. Comparing YOLOv3, YOLOv4 and YOLOv5 for autonomous landing spot detection in faulty UAVs[J]. Sensors, 2022, 22(2): 464. DOI URL
[16]	LearnOpenCV. YOLOv8 ultralytics: state-of-the-art YOLO models[EB/OL]. (2023-01-10) [2023-03-11]. https://learnopencv.com/ultralytics-YOLOv8/.
[17]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 7464-7475.
[18]	GE Z, LIU S T, WANG F, et al. YOLOX: exceeding YOLO series in 2021[EB/OL]. [2023-03-23]. https://arxiv.org/abs/2107.08430.pdf.
[19]	FENG C J, ZHONG Y J, GAO Y, et al. TOOD: task-aligned one-stage object detection[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 3490-3499.
[20]	JU R Y, CAI W M. Fracture detection in pediatric wrist trauma X-ray images using YOLOv8 algorithm[EB/OL]. [2023-05-11]. https://arxiv.org/abs/2304.05071.pdf.
[21]	SRINIVAS A, LIN T Y, PARMAR N, et al. Bottleneck transformers for visual recognition[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 16514-16524.
[22]	ZHANG J, ZHANG J S, ZHOU K X, et al. An improved YOLOv5-based underwater object-detection Framework[J]. Sensors, 2023, 23(7): 3693-3713. DOI URL
[23]	TAN M X, PANG R M, LE Q V. EfficientDet: scalable and efficient object detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 10778-10787.

隐私类别	相关子项
个人基本信息	生日、学/工号、家庭关系、社交关系、职业、教育经历、工作单位、吸烟、酗酒、性取向
生物识别信息	指纹、虹膜、人脸特征、身体裸露、基因信息
健康信息	病历、体检记录、住院记录、生病状况
证件信息	身份证、军官证、护照、驾驶证、户口本、学生证、准考证、车牌
通讯信息	电话号码、电子邮箱、通讯录、通信内容
位置信息	行踪轨迹、精准定位、住宿信息、车票
财产信息	银行账户、交易记录、借贷信息、房产信息、票据

隐私类别	相关子项
个人基本信息	生日、学/工号、家庭关系、社交关系、职业、教育经历、工作单位、吸烟、酗酒、性取向
生物识别信息	指纹、虹膜、人脸特征、身体裸露、基因信息
健康信息	病历、体检记录、住院记录、生病状况
证件信息	身份证、军官证、护照、驾驶证、户口本、学生证、准考证、车牌
通讯信息	电话号码、电子邮箱、通讯录、通信内容
位置信息	行踪轨迹、精准定位、住宿信息、车票
财产信息	银行账户、交易记录、借贷信息、房产信息、票据

隐私类别	标注类别	总计
个人基本信息	Personal	269
生物识别信息	Biometrics	307
健康信息	Health	300
证件信息	Certification	479
通讯信息	Communication	152
位置信息	Location	149
财产信息	Property	281

隐私类别	标注类别	总计
个人基本信息	Personal	269
生物识别信息	Biometrics	307
健康信息	Health	300
证件信息	Certification	479
通讯信息	Communication	152
位置信息	Location	149
财产信息	Property	281

模型	P	R	F1	mAP@.5	FPS
YOLOv3	0.732	0.674	0.709	0.717	47
YOLOv4	0.779	0.697	0.736	0.766	94
YOLOv5	0.875	0.762	0.815	0.850	133
YOLOv7	0.862	0.662	0.749	0.784	112
YOLOX	0.869	0.748	0.803	0.832	78
YOLOv8	0.863	0.834	0.848	0.896	226
本文模型	0.906	0.865	0.885	0.927	203