Multi-fitting detection method for transmission lines based on implicit spatial knowledge fusion

doi:10.11996/JG.j.2095-302X.2023050918

Abstract

Abstract:

To address the challenge of detecting tiny-size and dense occlusion objects in the task of multi-fitting detection for transmission lines, a transmission line multi-fitting detection method based on implicit spatial knowledge fusion was proposed. First, in order to mine the implicit spatial knowledge among transmission line fittings and assist the model in detection, the spatial box setting module and the spatial context extraction module were proposed to set the spatial box and extract the spatial context information. Then, the spatial context memory module was designed to filter and remember the spatial context information to aid the positioning of multi-fitting detection model. Finally, the post-processing part of the model was enhanced to further mitigate the issue of low detection accuracy stemming from dense occlusion by fittings. The experimental results demonstrated the efficacy of the proposed model in enhancing the detection of various kinds of fittings, especially those of tiny size and dense occlusion. Compared with the baseline model, the AP⁵⁰ evaluation index and the more stringent AP⁷⁵ evaluation index were increased by 3.5% and 5.7%, respectively. It laid a foundation for the application of fitting detection and further fault diagnosis.

Key words: transmission line, fitting detection, deep learning, implicit spatial knowledge, spatial context information

CLC Number:

TP391

ZHAI Yong-jie, GUO Cong-bin, WANG Qian-ming, ZHAO Kuan, BAI Yun-shan, ZHANG Ji. Multi-fitting detection method for transmission lines based on implicit spatial knowledge fusion[J]. Journal of Graphics, 2023, 44(5): 918-927.

Figures/Tables 11

References 24

[1]	李德海, 李丹丹. 基于视觉的输电线路自动监测技术综述[J]. 黑龙江电力, 2019, 41(6): 559-564.
	LI D H, LI D D. Overview of vision-based transmission line automatic monitoring technology[J]. Heilongjiang Electric Power, 2019, 41(6): 559-564. (in Chinese)
[2]	赵振兵, 蒋志钢, 李延旭, 等. 输电线路部件视觉缺陷检测综述[J]. 中国图象图形学报, 2021, 26(11): 2545-2560.
	ZHAO Z B, JIANG Z G, LI Y X, et al. Overview of visual defect detection of transmission line components[J]. Journal of Image and Graphics, 2021, 26(11): 2545-2560. (in Chinese)
[3]	NGUYEN V N, JENSSEN R, ROVERSO D. Automatic autonomous vision-based power line inspection: a review of current status and the potential role of deep learning[J]. International Journal of Electrical Power & Energy Systems, 2018, 99: 107-120. DOI URL
[4]	刘传洋, 吴一全. 基于深度学习的输电线路视觉检测方法研究进[J/OL]. 中国电机工程学报. [2022-12-06]. http://kns.cnki.net/kcms/detail/11.2107.TM.20220831.1053.002.html.
	LIU C Y, WU Y Q. Research progress on visual detection methods for transmission lines based on deep learning[J/OL]. Proceedings of the CSEE. [2022-12-06]. http://kns.cnki.net/kcms/detail/11.2107.TM.20220831.1053.002.html. (in Chinese)
[5]	黄郑, 王红星, 翟学锋, 等. 输电线路无人机自主巡检方法研究与应用[J]. 计算技术与自动化, 2021, 40(3): 157-161.
	HUANG Z, WANG H X, ZHAI X F, et al. Research and application of UAV autonomous inspection method for transmission line[J]. Computing Technology and Automation, 2021, 40(3): 157-161. (in Chinese)
[6]	金立军, 胡娟, 闫书佳. 基于图像的高压输电线间隔棒故障诊断方法[J]. 高电压技术, 2013, 39(5): 1040-1045.
	JIN L J, HU J, YAN S J. Method of spacer fault diagnose on transmission line based on image procession[J]. High Voltage Engineering, 2013, 39(5): 1040-1045. (in Chinese)
[7]	万林, 巫世晶, 谢福起, 等. 基于图像处理的输电线路耐张线夹监测系统[J]. 武汉大学学报: 工学版, 2020, 53(12): 1106-1111.
	WAN L, WU S J, XIE F Q, et al. Monitoring system of transmission line strain clamps based on image processing[J]. Engineering Journal of Wuhan University, 2020, 53(12): 1106-1111. (in Chinese)
[8]	王胜, 陈文, 匡小兵, 等. 一种基于多特征显著性融合的绝缘子区域检测与定位算法[J]. 计算机应用研究, 2020, 37(S2): 351-353.
	WANG S, CHEN W, KUANG X B, et al. Insulator area detection and localization algorithm based on multi-feature salient fusion[J]. Application Research of Computers, 2020, 37(S2): 351-353. (in Chinese)
[9]	戚银城, 江爱雪, 赵振兵, 等. 基于改进SSD模型的输电线路巡检图像金具检测方法[J]. 电测与仪表, 2019, 56(22): 7-12, 43.
	QI Y C, JIANG A X, ZHAO Z B, et al. Fittings detection method in patrol images of transmission line based on improved SSD[J]. Electrical Measurement & Instrumentation, 2019, 56(22): 7-12, 43. (in Chinese)
[10]	张永翔, 吴功平, 刘中云, 等. 基于YOLOv3网络的输电线路防震锤和线夹检测迁移学习[J]. 计算机应用, 2020, 40(S2): 188-194.
	ZHANG Y X, WU G P, LIU Z Y, et al. Transfer learning of transmission line damper and clamp detection based on YOLOv3 network[J]. Journal of Computer Applications, 2020, 40(S2): 188-194. (in Chinese)
[11]	李鑫, 刘帅男, 杨桢, 等. 基于改进Cascade R-CNN的输电线路多目标检测[J]. 电子测量与仪器学报, 2021, 35(10): 24-32.
	LI X, LIU S N, YANG Z, et al. Multi-target detection of transmission lines based on improved cascade R-CNN[J]. Journal of Electronic Measurement and Instrumentation, 2021, 35(10): 24-32. (in Chinese)
[12]	易继禹, 陈慈发, 龚国强. 基于改进Faster RCNN的输电线路航拍绝缘子检测[J]. 计算机工程, 2021, 47(6): 292-298, 304. DOI
	YI J Y, CHEN C F, GONG G Q. Aerial insulator detection of transmission line based on improved faster RCNN[J]. Computer Engineering, 2021, 47(6): 292-298, 304. (in Chinese) DOI
[13]	WU Y, CHEN Y P, YUAN L, et al. Rethinking classification and localization for object detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 10186-10195.
[14]	BODLA N, SINGH B, CHELLAPPA R, et al. Soft-NMS--improving object detection with one line of code[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 5561-5569.
[15]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[16]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[17]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 2117-2125.
[18]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2961-2969.
[19]	CHO K, VAN MERRIËNBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[EB/OL]. [2023-8-16]. https://arxiv.org/abs/1406.1078.
[20]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// 2016 European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[21]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2980-2988.
[22]	CHEN Q, WANG Y M, YANG T, et al. You only look one-level feature[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13039-13048.
[23]	ZHANG H K, CHANG H, MA B P, et al. Dynamic R-CNN: towards high quality object detection via dynamic training[C]// 2020 European Conference on Computer Vision. Cham: Springer, 2020: 260-275.
[24]	PANG J M, CHEN K, SHI J P, et al. Libra R-CNN: towards balanced learning for object detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 821-830.

基线模型	隐含空间知识融合	改进后处理	AP⁵⁰	AP⁷⁵
√	-	-	77.5	41.9
√	√	-	81.0	44.9
√	-	√	78.2	45.0
√	√	√	81.0	47.6

基线模型	隐含空间知识融合	改进后处理	AP⁵⁰	AP⁷⁵
√	-	-	77.5	41.9
√	√	-	81.0	44.9
√	-	√	78.2	45.0
√	√	√	81.0	47.6

α	AP^50-95	AP⁵⁰	AP⁷⁵	AR¹	AR¹⁰	AR¹⁰⁰
0.10	45.4	80.1	45.7	26.5	56.0	59.1
0.15	45.3	80.0	46.1	26.3	56.2	58.9
0.20	46.1	80.1	47.9	27.3	56.3	59.1
0.25	46.6	81.0	47.6	27.3	56.9	59.6
0.30	45.6	80.2	47.5	26.8	55.6	58.7

α	AP^50-95	AP⁵⁰	AP⁷⁵	AR¹	AR¹⁰	AR¹⁰⁰
0.10	45.4	80.1	45.7	26.5	56.0	59.1
0.15	45.3	80.0	46.1	26.3	56.2	58.9
0.20	46.1	80.1	47.9	27.3	56.3	59.1
0.25	46.6	81.0	47.6	27.3	56.9	59.6
0.30	45.6	80.2	47.5	26.8	55.6	58.7

方法	AP^50-95	AP⁵⁰	AP⁷⁵	AP^S	AP^M	AP^L	AR¹	AR¹⁰	AR¹⁰⁰	AR^S	AR^M	AR^L	Time (s/im)
SSD	33.5	65.8	30.4	13.9	14.0	37.5	22.4	41.5	42.6	20.6	22.1	46.8	0.008
RetinaNet	35.5	66.7	33.6	9.1	25.7	38.6	22.5	46.8	48.1	15.4	32.4	51.8	0.050
YOLOF	43.5	77.1	43.4	14.1	27.1	47.6	26.5	51.1	51.7	14.0	34.0	56.4	0.033
Dynamic R-CNN	43.7	75.1	45.4	19.6	34.0	46.8	26.4	51.8	52.2	19.6	39.2	55.9	0.041
Libra R-CNN	45.3	80.1	45.7	14.1	32.3	48.3	27.6	53.9	54.9	17.1	39.0	58.4	0.042
基线模型	42.1	77.5	41.9	14.9	32.9	45.0	25.7	50.1	50.9	15.1	39.4	54.0	0.069
本文模型	46.6	81.0	47.6	24.1	37.1	49.7	27.3	56.9	59.6	24.4	46.8	63.1	0.190