基于改进YOLOv8与语义知识融合的金具缺陷检测方法研究

doi:10.11996/JG.j.2095-302X.2024050979

图学学报 ›› 2024, Vol. 45 ›› Issue (5): 979-986.DOI: 10.11996/JG.j.2095-302X.2024050979

• 图像处理与计算机视觉 • 上一篇下一篇

基于改进YOLOv8与语义知识融合的金具缺陷检测方法研究

李刚¹^,²(), 蔡泽浩¹, 孙华勋³, 赵振兵²

1.华北电力大学计算机系，河北保定 071003
2.复杂能源系统智能计算教育部工程研究中心，河北保定 071003
3.中国华电集团有限公司，北京 100031

收稿日期:2024-06-10 修回日期:2024-09-05 出版日期:2024-10-31 发布日期:2024-10-31
第一作者:李刚(1980-)，男，副教授，博士。主要研究方向为电力人工智能、电力视觉。E-mail：ququ_er2003@126.com
基金资助:
国家自然科学基金项目(61871182);国家自然科学基金项目(U21A20486)

Research on defect detection of transmission line fittings based on improved YOLOv8 and semantic knowledge fusion

LI Gang¹^,²(), CAI Zehao¹, SUN Huaxun³, ZHAO Zhenbing²

1. Department of Computer, North China Electric Power University, Baoding Hebei 071003, China
2. Engineering Research Center of Intelligent Computing for Complex Energy Systems, Ministry of Education, Baoding Hebei 071003, China
3. China Huadian Group Co., Ltd., Beijing 100031, China

Received:2024-06-10 Revised:2024-09-05 Published:2024-10-31 Online:2024-10-31
First author：LI Gang (1980-), associate professor, Ph.D. His main research interests cover AI for power system and electric power vision. E-mail：ququ_er2003@126.com
Supported by:
National Natural Science Foundation of China(61871182);National Natural Science Foundation of China(U21A20486)

摘要/Abstract

摘要：

针对输电线路螺栓金具缺陷检测任务中存在的缺陷样本类间分布不均、缺陷微小特征提取困难等问题，提出基于改进YOLOv8和语义知识融合的输电线路螺栓缺陷检测方法。首先，通过深入分析数据样本中螺栓金具缺陷种类与该螺栓承载金具种类之间的关系，完成语义关联构建工作；之后，在YOLOv8模型Neck部分引入BiFusion和RepBlock模块，增强模型的特征提取能力；其次，使用改进的融合语义知识校正权重的Loss函数，进一步提高训练模型的准确性，减少误检的发生；最后，分别完成基线选取实验、消融实验、超参数调整实验以及对比实验。实验结果表明，相较于Baseline模型，改进YOLOv8方法在平均精确率(mAP)上提升了4.0%，在关键少样本类精确率上提升了24.6%，可有效提高输电线路螺栓金具缺陷检测的效果，该语义关联构建及语义知识融合方法具有一定的泛用性，为输电线路无人机智能巡检领域提供了新的方法支持。

关键词: 无人机巡检, 输电线路金具, 螺栓缺陷检测, 语义信息融合, YOLOv8

Abstract:

To address issues such as the uneven distribution of defect samples among classes and the difficulty in extracting tiny features of defects in the task of detecting defects of transmission line bolts and fixtures, a defect detection method for transmission line bolts and fixtures was proposed based on the improvement of YOLOv8 and semantic knowledge fusion. First, the semantic correlation was established by analyzing the relationship between the defective types of bolt fittings in the data samples and the types of fittings carried by that bolt. Then, the BiFusion and RepBlock modules were introduced into the Neck part of the YOLOv8 model to enhance its feature extraction capability. Second, the Loss function of the weights was corrected using an improved fusion of semantic knowledge, further improving the accuracy of the training model and reducing the occurrence of misdetection. Finally, baseline selection experiments, ablation experiments, hyper-parameter experiments, and comparative experiments were conducted, respectively. The experimental results showed that compared with the Baseline model, the improved YOLOv8 method increased the mean average precision (mAP) by 4.0% and improved the accuracy of the key less sample classes by 24.6%, effectively enhancing the defect detection performance for transmission line bolted fittings. The proposed semantic correlation establishment and semantic knowledge fusion method also demonstrated a certain degree of generalizability, providing new methodological support for UAV-based intelligent inspection of transmission lines.

Key words: UAV inspection, transmission line fittings, bolt defect detection, semantic information fusion, YOLOv8

中图分类号:

TP391

李刚, 蔡泽浩, 孙华勋, 赵振兵. 基于改进YOLOv8与语义知识融合的金具缺陷检测方法研究[J]. 图学学报, 2024, 45(5): 979-986.

LI Gang, CAI Zehao, SUN Huaxun, ZHAO Zhenbing. Research on defect detection of transmission line fittings based on improved YOLOv8 and semantic knowledge fusion[J]. Journal of Graphics, 2024, 45(5): 979-986.

图/表 11

参考文献 19

[1]	赵振兵, 张薇, 翟永杰, 等. 电力视觉技术的概念、研究现状与展望[J]. 电力科学与工程, 2020, 36(1): 1-8. DOI
	ZHAO Z B, ZHANG W, ZHAI Y J, et al. Concept, research status and prospect of electric power vision technology[J]. Electric Power Science and Engineering, 2020, 36(1): 1-8 (in Chinese).
[2]	赵振兵, 蒋志钢, 李延旭, 等. 输电线路部件视觉缺陷检测综述[J]. 中国图象图形学报, 2021, 26(11): 2545-2560.
	ZHAO Z B, JIANG Z G, LI Y X, et al. Overview of visual defect detection of transmission line components[J]. Journal of Image and Graphics, 2021, 26(11): 2545-2560 (in Chinese).
[3]	赵振兵, 王睿, 赵文清, 等. 基于图知识推理的输电线路缺销螺栓识别方法[J]. 智能系统学报, 2023, 18(2): 372-380.
	ZHAO Z B, WANG R, ZHAO W Q, et al. Pin-missing bolts recognition method for transmission lines based on graph knowledge reasoning[J]. CAAI Transactions on Intelligent Systems, 2023, 18(2): 372-380 (in Chinese).
[4]	翟永杰, 郭聪彬, 王乾铭, 等. 基于隐含空间知识融合的输电线路多金具检测方法[J]. 图学学报, 2023, 44(5): 918-927. DOI
	ZHAI Y J, GUO C B, WANG Q M, et al. Multi-fitting detection method for transmission lines based on implicit spatial knowledge fusion[J]. Journal of Graphics, 2023, 44(5): 918-927 (in Chinese).
[5]	王亚茹, 冯利龙, 宋晓轲, 等. TFD-YOLOv8: 一种用于输电线路的异物检测方法[EB/OL]. [2024-07-24]. http://kns.cnki.net/kcms/detail/10.1034.T.20240703.1534.002.html.
	WANG YR, FENG LL, SONG XK, et al. TFD-YOLOv8: a transmission line foreign object detection method[EB/OL]. [2024-07-24]. http://kns.cnki.net/kcms/detail/10.1034.T.20240 703.1534.002.html (in Chinese).
[6]	LIU S, LIU L W, SHANG F, et al. Research and implementation of birds and floating objects target detection technology in transmission lines[C]// 2022 10th IEEE Joint International Information Technology and Artificial Intelligence Conference. New York: IEEE Press, 2022: 2528-2532.
[7]	BI Z Q, JING L N, SUN C, et al. YOLOX++ for transmission line abnormal target detection[J]. IEEE Access, 2023, 11: 38157-38167.
[8]	张珂, 周睿恒, 石超君, 等. 自监督E-Swin的输电线路金具检测[J]. 中国图象图形学报, 2023, 28(10): 3064-3076.
	ZHANG K, ZHOU R H, SHI C J, et al. Self-supervised E-Swin based transmission line fittings detection[J]. Journal of Image and Graphics, 2023, 28(10): 3064-3076 (in Chinese).
[9]	CHEN D Q, GUOX H, HUANGP, et al. Safety distance analysis of 500kV transmission line tower UAV patrol inspection[J]. IEEE Letters on Electromagnetic Compatibility Practice and Applications, 2020, 2(4): 124-128.
[10]	苟军年, 杜愫愫, 王世铎, 等. 轻量化特征融合的CenterNet输电线路绝缘子自爆缺陷检测[J]. 北京航空航天大学学报, 2024, 50(7): 2161-2171.
	GOU J N, DU SS, WANG S D, et al. Insulator self-explosion detection in transmission line based on CenterNet fusing lightweight features[J]. Journal of Beijing University of Aeronautics and Astronautics, 2024, 50(7): 2161-2171 (in Chinese).
[11]	YANG Q F, ZHANG Z Q, YAN L, et al. Lightweight Bird's nest location recognition method based on YOLOv4- tiny[C]// 2021 IEEE International Conference on Electrical Engineering and Mechatronics Technology. New York: IEEE Press, 2021: 402-405.
[12]	赵振兵, 熊静, 徐厚东, 等. 融合结构推理的深度模型输电线路金具及其缺陷检测[J]. 高电压技术, 2023, 49(8): 3346-3353.
	ZHAO Z B, XIONG J, XU H D, et al. Integrating structural reasoning for deep model transmission line fittings and their defects detection[J]. High Voltage Engineering, 2023, 49(8): 3346-3353 (in Chinese).
[13]	戚银城, 耿劭锋, 赵振兵, 等. 基于特征迁移的螺栓图像超分辨率处理方法[J]. 智能系统学报, 2023, 18(4): 858-866.
	QIY C, GENG S F, ZHAO Z B, et al. A method for super resolution processing of bolt image based on feature transfer[J]. CAAI Transactions on Intelligent Systems, 2023, 18(4): 858-866 (in Chinese).
[14]	李刚, 张运涛, 汪文凯, 等. 采用DETR与先验知识融合的输电线路螺栓缺陷检测方法[J]. 图学学报, 2023, 44(3): 438-447. DOI
	LI G, ZHANG Y T, WANG W K, et al. Defect detection method of transmission line bolts based on DETR and prior knowledge fusion[J]. Journal of Graphics, 2023, 44(3): 438-447 (in Chinese). DOI
[15]	刘起源, 路锦正, 黄炳森. 基于特征融合和损失优化的点云语义分割网络[J]. 计算机技术与发展, 2024, 34(5): 66-72.
	LIUQ Y, LU J Z, HUANG B S. Point cloud semantic segmentation network based on feature fusion and loss optimization[J]. Computer Technology and Development, 2024, 34(5): 66-72 (in Chinese).
[16]	骆俊锋. 结合高层语义与底层颜色特征的图像检索方法[J]. 电子元器件与信息技术, 2021, 5(3): 188-190.
	LUO J F. Image retrieval method combining high-level semantics and low-level color features[J]. Electronic Component and Information Technology, 2021, 5(3): 188-190 (in Chinese).
[17]	禹鑫燚, 林密, 卢江平, 等. 基于向量叉乘标签分配的遥感图像目标检测算法[J]. 高技术通讯, 2024, 34(2): 132-142.
	YU X Y, LIN M, LU J P, et al. Remote sensing object detection algorithm based on vector cross product label assignment[J]. Chinese High Technology Letters, 2024, 34(2): 132-142 (in Chinese).
[18]	CHEN P Y, HSIEH J W, WANG C Y, et al. Residual Bi-fusion feature pyramid network for accurate single-shot object detection[EB/OL]. [2024-04-10]. https://arxiv.org/abs/1911.12051.
[19]	LI C Y, LI L L, JIANG H L, et al. YOLOv6: a single-stage object detection framework for industrial applications[EB/OL]. [2024-01-12]. https://arxiv.org/abs/2209.02976.

名称	型号
操作系统	Windows 10家庭版
显示适配器	GeForce RTX 4060 12 G
CUDA	11.6
Python	3.8
Pytorch	1.13.1
Torchvision	0.14.1
Batch size	8
Epoch	200

名称	型号
操作系统	Windows 10家庭版
显示适配器	GeForce RTX 4060 12 G
CUDA	11.6
Python	3.8
Pytorch	1.13.1
Torchvision	0.14.1
Batch size	8
Epoch	200

缺陷种类	样本数量	标签名称
正常	263	B@front@normal
	527	A@front@normal
	414	A@side_B@normal
	392	A@side_A@normal
销子缺失	688	A@side_AB@error_A
销子缺失	215	A@front@error_A
销子失效	39	A@side_B@error_B
	44	A@front@error_B
	180	B@front@error_B
垫片缺失	134	B@front@error_C

缺陷种类	样本数量	标签名称
正常	263	B@front@normal
	527	A@front@normal
	414	A@side_B@normal
	392	A@side_A@normal
销子缺失	688	A@side_AB@error_A
销子缺失	215	A@front@error_A
销子失效	39	A@side_B@error_B
	44	A@front@error_B
	180	B@front@error_B
垫片缺失	134	B@front@error_C

Model	mAP/%	FPS	GFLOPs
YOLOv8n	81.5	342.4	12.0
YOLOv8s	84.8	154.8	42.5
YOLOv8m	84.8	70.0	110.0
YOLOv8l	84.0	45.7	220.2
YOLOv8x	85.9	30.1	343.7

基于改进YOLOv8与语义知识融合的金具缺陷检测方法研究

Research on defect detection of transmission line fittings based on improved YOLOv8 and semantic knowledge fusion

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 19

相关文章 14

编辑推荐

Metrics

本文评价

标签名称	Wire1	Wire2	Wire3	Insulator1	Insulator2	Insulator3	Insulator4	Insulator5	Insulator6	U
B@front@normal	260	0	3	0	0	0	0	0	0	0
A@front@normal	156	48	0	103	118	13	25	47	3	14
A@sideB@normal	28	24	1	104	18	128	18	54	27	12
A@sideA@normal	10	12	0	58	2	103	10	19	12	166
A@sideAB@errorA	520	1	4	23	20	76	4	6	7	27
A@front@error_A	94	41	0	53	12	3	2	1	0	9
A@sideB@error_B	2	1	0	10	3	14	1	5	3	0
A@front@error_B	6	4	0	19	9	1	3	2	0	0
B@front@error_B	180	0	0	0	0	0	0	0	0	0
B@front@error_C	106	0	8	0	0	0	0	0	0	0

YOLOV8s	改进1	改进2	AP/%										mAP/%
YOLOV8s	改进1	改进2	类1	类2	类3	类4	类5	类6	类7	类8	类9	类10	mAP/%
√	-	-	99.5	89.4	98.4	92.9	89.4	81.5	91.4	40.0	65.5	99.5	84.8
√	√	-	98.2	86.5	96.5	95.1	87.0	81.0	93.3	48.3	72.0	99.5	85.8
√	-	√	98.1	87.3	97.7	95.1	90.0	80.9	97.5	36.5	79.4	99.5	86.2
√	√	√	98.5	87.8	99.0	97.1	89.5	87.5	90.9	55.2	65.4	99.5	87.0

α取值	AP/%										mAP/%
α取值	类1	类2	类3	类4	类5	类6	类7	类8	类9	类10	mAP/%
2.7	99.5	87.7	98.0	95.3	87.0	84.7	95.3	43.5	58.3	99.5	84.9
2.8	96.9	91.6	96.3	95.4	92.8	87.7	97.0	47.7	64.2	99.5	86.9
2.9	99.3	90.0	98.1	95.2	90.4	87.2	99.5	47.9	65.5	99.5	87.3
3.0	98.4	90.5	98.9	95.8	91.9	91.2	90.7	64.6	66.9	99.5	88.8
3.5	96.7	92.5	99.3	96.7	93.8	86.9	90.3	47.1	65.0	99.5	86.8
4.0	96.7	89.6	98.6	97.5	94.2	85.6	93.0	43.3	74.6	99.5	87.3

模型名称	AP/%										mAP/%
模型名称	类1	类2	类3	类4	类5	类6	类7	类8	类9	类10	mAP/%
YOLOv5	97.0	85.2	96.4	90.0	88.9	86.8	90.6	54	63.7	99.5	85.2
YOLOv5改进后	97.2	85.3	96.7	92.6	87.9	80.6	91.5	65.8	67.4	99.5	86.4
YOLOv3	98.0	91.3	98.1	94.7	87.0	79.9	96.5	32.6	64.6	99.5	84.2
YOLOv3改进后	99.0	88.9	96.8	94.0	93.1	83.0	99.5	41.7	70.0	99.5	86.5
YOLOv8+AIFI	98.0	86.3	98.7	93.6	92.1	85.6	93.2	19.9	67.4	99.5	83.4
YOLOv8+DAttention	99.1	86.3	96.3	95.3	88.5	81.1	97.9	16.6	35.4	99.5	79.6
YOLOv8+GoldYolo	93.0	88.8	97.3	93.1	84.4	72.8	93.2	28.9	62.9	99.5	81.4

[1]	李珍峰, 符世琛, 徐乐, 孟博, 张昕, 秦建军. 基于MBI-YOLOv8的煤矸石目标检测算法研究[J]. 图学学报, 2024, 45(6): 1301-1312.
[2]	闫建红, 冉同霄. 基于YOLOv8的轻量化无人机图像目标检测算法[J]. 图学学报, 2024, 45(6): 1328-1337.
[3]	胡凤阔, 叶兰, 谭显峰, 张钦展, 胡志新, 方清, 王磊, 满孝锋. 一种基于改进YOLOv8的轻量化路面病害检测算法[J]. 图学学报, 2024, 45(5): 892-900.
[4]	王亚茹, 冯利龙, 宋晓轲, 屈卓, 杨珂, 王乾铭, 翟永杰. TFD-YOLOv8：一种用于输电线路的异物检测方法[J]. 图学学报, 2024, 45(5): 901-912.
[5]	刘义艳, 郝婷楠, 贺晨, 常英杰. 基于DBBR-YOLO的光伏电池表面缺陷检测[J]. 图学学报, 2024, 45(5): 913-921.
[6]	胡欣, 常娅姝, 秦皓, 肖剑, 程鸿亮. 基于改进YOLOv8和GMM图像点集匹配的双目测距方法[J]. 图学学报, 2024, 45(4): 714-725.
[7]	牛为华, 郭迅. 基于改进YOLOv8的船舰遥感图像旋转目标检测算法[J]. 图学学报, 2024, 45(4): 726-735.
[8]	曾志超, 徐玥, 王景玉, 叶元龙, 黄志开, 王欢. 基于SOE-YOLO轻量化的水面目标检测算法[J]. 图学学报, 2024, 45(4): 736-744.
[9]	武兵, 田莹. 基于注意力机制的多尺度道路损伤检测算法研究[J]. 图学学报, 2024, 45(4): 770-778.
[10]	朱强军, 胡斌, 汪慧兰, 王杨. 基于轻量化YOLOv8s交通标志的检测[J]. 图学学报, 2024, 45(3): 422-432.
[11]	崔克彬, 焦静颐. 基于MCB-FAH-YOLOv8的钢材表面缺陷检测算法[J]. 图学学报, 2024, 45(1): 112-125.
[12]	王大阜, 王静, 石宇凯, 邓志文, 贾志勇. 基于深度迁移学习的图像隐私目标检测研究[J]. 图学学报, 2023, 44(6): 1112-1120.
[13]	高昂, 梁兴柱, 夏晨星, 张春炯. 一种改进YOLOv8的密集行人检测算法[J]. 图学学报, 2023, 44(5): 890-898.
[14]	李刚, 张运涛, 汪文凯, 张东阳. 采用DETR与先验知识融合的输电线路螺栓缺陷检测方法[J]. 图学学报, 2023, 44(3): 438-447.