Research on defect detection of transmission line fittings based on improved YOLOv8 and semantic knowledge fusion

doi:10.11996/JG.j.2095-302X.2024050979

Abstract

Abstract:

To address issues such as the uneven distribution of defect samples among classes and the difficulty in extracting tiny features of defects in the task of detecting defects of transmission line bolts and fixtures, a defect detection method for transmission line bolts and fixtures was proposed based on the improvement of YOLOv8 and semantic knowledge fusion. First, the semantic correlation was established by analyzing the relationship between the defective types of bolt fittings in the data samples and the types of fittings carried by that bolt. Then, the BiFusion and RepBlock modules were introduced into the Neck part of the YOLOv8 model to enhance its feature extraction capability. Second, the Loss function of the weights was corrected using an improved fusion of semantic knowledge, further improving the accuracy of the training model and reducing the occurrence of misdetection. Finally, baseline selection experiments, ablation experiments, hyper-parameter experiments, and comparative experiments were conducted, respectively. The experimental results showed that compared with the Baseline model, the improved YOLOv8 method increased the mean average precision (mAP) by 4.0% and improved the accuracy of the key less sample classes by 24.6%, effectively enhancing the defect detection performance for transmission line bolted fittings. The proposed semantic correlation establishment and semantic knowledge fusion method also demonstrated a certain degree of generalizability, providing new methodological support for UAV-based intelligent inspection of transmission lines.

Key words: UAV inspection, transmission line fittings, bolt defect detection, semantic information fusion, YOLOv8

CLC Number:

TP391

LI Gang, CAI Zehao, SUN Huaxun, ZHAO Zhenbing. Research on defect detection of transmission line fittings based on improved YOLOv8 and semantic knowledge fusion[J]. Journal of Graphics, 2024, 45(5): 979-986.

Figures/Tables 11

References 19

[1]	赵振兵, 张薇, 翟永杰, 等. 电力视觉技术的概念、研究现状与展望[J]. 电力科学与工程, 2020, 36(1): 1-8. DOI
	ZHAO Z B, ZHANG W, ZHAI Y J, et al. Concept, research status and prospect of electric power vision technology[J]. Electric Power Science and Engineering, 2020, 36(1): 1-8 (in Chinese).
[2]	赵振兵, 蒋志钢, 李延旭, 等. 输电线路部件视觉缺陷检测综述[J]. 中国图象图形学报, 2021, 26(11): 2545-2560.
	ZHAO Z B, JIANG Z G, LI Y X, et al. Overview of visual defect detection of transmission line components[J]. Journal of Image and Graphics, 2021, 26(11): 2545-2560 (in Chinese).
[3]	赵振兵, 王睿, 赵文清, 等. 基于图知识推理的输电线路缺销螺栓识别方法[J]. 智能系统学报, 2023, 18(2): 372-380.
	ZHAO Z B, WANG R, ZHAO W Q, et al. Pin-missing bolts recognition method for transmission lines based on graph knowledge reasoning[J]. CAAI Transactions on Intelligent Systems, 2023, 18(2): 372-380 (in Chinese).
[4]	翟永杰, 郭聪彬, 王乾铭, 等. 基于隐含空间知识融合的输电线路多金具检测方法[J]. 图学学报, 2023, 44(5): 918-927. DOI
	ZHAI Y J, GUO C B, WANG Q M, et al. Multi-fitting detection method for transmission lines based on implicit spatial knowledge fusion[J]. Journal of Graphics, 2023, 44(5): 918-927 (in Chinese).
[5]	王亚茹, 冯利龙, 宋晓轲, 等. TFD-YOLOv8: 一种用于输电线路的异物检测方法[EB/OL]. [2024-07-24]. http://kns.cnki.net/kcms/detail/10.1034.T.20240703.1534.002.html.
	WANG YR, FENG LL, SONG XK, et al. TFD-YOLOv8: a transmission line foreign object detection method[EB/OL]. [2024-07-24]. http://kns.cnki.net/kcms/detail/10.1034.T.20240 703.1534.002.html (in Chinese).
[6]	LIU S, LIU L W, SHANG F, et al. Research and implementation of birds and floating objects target detection technology in transmission lines[C]// 2022 10th IEEE Joint International Information Technology and Artificial Intelligence Conference. New York: IEEE Press, 2022: 2528-2532.
[7]	BI Z Q, JING L N, SUN C, et al. YOLOX++ for transmission line abnormal target detection[J]. IEEE Access, 2023, 11: 38157-38167.
[8]	张珂, 周睿恒, 石超君, 等. 自监督E-Swin的输电线路金具检测[J]. 中国图象图形学报, 2023, 28(10): 3064-3076.
	ZHANG K, ZHOU R H, SHI C J, et al. Self-supervised E-Swin based transmission line fittings detection[J]. Journal of Image and Graphics, 2023, 28(10): 3064-3076 (in Chinese).
[9]	CHEN D Q, GUOX H, HUANGP, et al. Safety distance analysis of 500kV transmission line tower UAV patrol inspection[J]. IEEE Letters on Electromagnetic Compatibility Practice and Applications, 2020, 2(4): 124-128.
[10]	苟军年, 杜愫愫, 王世铎, 等. 轻量化特征融合的CenterNet输电线路绝缘子自爆缺陷检测[J]. 北京航空航天大学学报, 2024, 50(7): 2161-2171.
	GOU J N, DU SS, WANG S D, et al. Insulator self-explosion detection in transmission line based on CenterNet fusing lightweight features[J]. Journal of Beijing University of Aeronautics and Astronautics, 2024, 50(7): 2161-2171 (in Chinese).
[11]	YANG Q F, ZHANG Z Q, YAN L, et al. Lightweight Bird's nest location recognition method based on YOLOv4- tiny[C]// 2021 IEEE International Conference on Electrical Engineering and Mechatronics Technology. New York: IEEE Press, 2021: 402-405.
[12]	赵振兵, 熊静, 徐厚东, 等. 融合结构推理的深度模型输电线路金具及其缺陷检测[J]. 高电压技术, 2023, 49(8): 3346-3353.
	ZHAO Z B, XIONG J, XU H D, et al. Integrating structural reasoning for deep model transmission line fittings and their defects detection[J]. High Voltage Engineering, 2023, 49(8): 3346-3353 (in Chinese).
[13]	戚银城, 耿劭锋, 赵振兵, 等. 基于特征迁移的螺栓图像超分辨率处理方法[J]. 智能系统学报, 2023, 18(4): 858-866.
	QIY C, GENG S F, ZHAO Z B, et al. A method for super resolution processing of bolt image based on feature transfer[J]. CAAI Transactions on Intelligent Systems, 2023, 18(4): 858-866 (in Chinese).
[14]	李刚, 张运涛, 汪文凯, 等. 采用DETR与先验知识融合的输电线路螺栓缺陷检测方法[J]. 图学学报, 2023, 44(3): 438-447. DOI
	LI G, ZHANG Y T, WANG W K, et al. Defect detection method of transmission line bolts based on DETR and prior knowledge fusion[J]. Journal of Graphics, 2023, 44(3): 438-447 (in Chinese). DOI
[15]	刘起源, 路锦正, 黄炳森. 基于特征融合和损失优化的点云语义分割网络[J]. 计算机技术与发展, 2024, 34(5): 66-72.
	LIUQ Y, LU J Z, HUANG B S. Point cloud semantic segmentation network based on feature fusion and loss optimization[J]. Computer Technology and Development, 2024, 34(5): 66-72 (in Chinese).
[16]	骆俊锋. 结合高层语义与底层颜色特征的图像检索方法[J]. 电子元器件与信息技术, 2021, 5(3): 188-190.
	LUO J F. Image retrieval method combining high-level semantics and low-level color features[J]. Electronic Component and Information Technology, 2021, 5(3): 188-190 (in Chinese).
[17]	禹鑫燚, 林密, 卢江平, 等. 基于向量叉乘标签分配的遥感图像目标检测算法[J]. 高技术通讯, 2024, 34(2): 132-142.
	YU X Y, LIN M, LU J P, et al. Remote sensing object detection algorithm based on vector cross product label assignment[J]. Chinese High Technology Letters, 2024, 34(2): 132-142 (in Chinese).
[18]	CHEN P Y, HSIEH J W, WANG C Y, et al. Residual Bi-fusion feature pyramid network for accurate single-shot object detection[EB/OL]. [2024-04-10]. https://arxiv.org/abs/1911.12051.
[19]	LI C Y, LI L L, JIANG H L, et al. YOLOv6: a single-stage object detection framework for industrial applications[EB/OL]. [2024-01-12]. https://arxiv.org/abs/2209.02976.

标签名称	Wire1	Wire2	Wire3	Insulator1	Insulator2	Insulator3	Insulator4	Insulator5	Insulator6	U
B@front@normal	260	0	3	0	0	0	0	0	0	0
A@front@normal	156	48	0	103	118	13	25	47	3	14
A@sideB@normal	28	24	1	104	18	128	18	54	27	12
A@sideA@normal	10	12	0	58	2	103	10	19	12	166
A@sideAB@errorA	520	1	4	23	20	76	4	6	7	27
A@front@error_A	94	41	0	53	12	3	2	1	0	9
A@sideB@error_B	2	1	0	10	3	14	1	5	3	0
A@front@error_B	6	4	0	19	9	1	3	2	0	0
B@front@error_B	180	0	0	0	0	0	0	0	0	0
B@front@error_C	106	0	8	0	0	0	0	0	0	0

标签名称	Wire1	Wire2	Wire3	Insulator1	Insulator2	Insulator3	Insulator4	Insulator5	Insulator6	U
B@front@normal	260	0	3	0	0	0	0	0	0	0
A@front@normal	156	48	0	103	118	13	25	47	3	14
A@sideB@normal	28	24	1	104	18	128	18	54	27	12
A@sideA@normal	10	12	0	58	2	103	10	19	12	166
A@sideAB@errorA	520	1	4	23	20	76	4	6	7	27
A@front@error_A	94	41	0	53	12	3	2	1	0	9
A@sideB@error_B	2	1	0	10	3	14	1	5	3	0
A@front@error_B	6	4	0	19	9	1	3	2	0	0
B@front@error_B	180	0	0	0	0	0	0	0	0	0
B@front@error_C	106	0	8	0	0	0	0	0	0	0

名称	型号
操作系统	Windows 10家庭版
显示适配器	GeForce RTX 4060 12 G
CUDA	11.6
Python	3.8
Pytorch	1.13.1
Torchvision	0.14.1
Batch size	8
Epoch	200

名称	型号
操作系统	Windows 10家庭版
显示适配器	GeForce RTX 4060 12 G
CUDA	11.6
Python	3.8
Pytorch	1.13.1
Torchvision	0.14.1
Batch size	8
Epoch	200

缺陷种类	样本数量	标签名称
正常	263	B@front@normal
	527	A@front@normal
	414	A@side_B@normal
	392	A@side_A@normal
销子缺失	688	A@side_AB@error_A
销子缺失	215	A@front@error_A
销子失效	39	A@side_B@error_B
	44	A@front@error_B
	180	B@front@error_B
垫片缺失	134	B@front@error_C