基于隐含空间知识融合的输电线路多金具检测方法

doi:10.11996/JG.j.2095-302X.2023050918

图学学报 ›› 2023, Vol. 44 ›› Issue (5): 918-927.DOI: 10.11996/JG.j.2095-302X.2023050918

• 图像处理与计算机视觉 • 上一篇下一篇

基于隐含空间知识融合的输电线路多金具检测方法

翟永杰¹(), 郭聪彬¹, 王乾铭¹, 赵宽¹, 白云山¹, 张冀²()

1.华北电力大学自动化系，河北保定 071003
2.华北电力大学计算机系，河北保定 071003

收稿日期:2023-04-24 接受日期:2023-08-25 出版日期:2023-10-31 发布日期:2023-10-31
通讯作者: 张冀(1972-)，男，副教授，博士。主要研究方向为计算机视觉、图像处理和信息融合。E-mail：zhangji@ncepu.edu.cn
作者简介:翟永杰(1972-)，男，教授，博士。主要研究方向为模式识别和数字图像处理。E-mail：zhaiyongjie@ncepu.edu.cn
基金资助:
国家自然科学基金项目(U21A20486);国家自然科学基金项目(61871182);中央高校基本科研业务费专项资金项目(2021MS081);中央高校基本科研业务费专项资金项目(2023JC006)

Multi-fitting detection method for transmission lines based on implicit spatial knowledge fusion

ZHAI Yong-jie¹(), GUO Cong-bin¹, WANG Qian-ming¹, ZHAO Kuan¹, BAI Yun-shan¹, ZHANG Ji²()

1. Department of Automation, North China Electric Power University, Baoding Hebei 071003, China
2. Department of Computer Science, North China Electric Power University, Baoding Hebei 071003, China

Received:2023-04-24 Accepted:2023-08-25 Online:2023-10-31 Published:2023-10-31
Contact: ZHANG Ji (1972-), associate professor, Ph.D. His main research interests cover computer vision, image processing and information fusion. E-mail：zhangji@ncepu.edu.cn
About author:ZHAI Yong-jie (1972-), professor, Ph.D. His main research interests cover pattern recognition and digital image processing. E-mail：zhaiyongjie@ncepu.edu.cn
Supported by:
National Natural Science Foundation of China(U21A20486);National Natural Science Foundation of China(61871182);Fundamental Research Funds for the Central Universities(2021MS081);Fundamental Research Funds for the Central Universities(2023JC006)

摘要/Abstract

摘要：

针对输电线路多金具检测任务中的小目标问题和密集遮挡问题，提出基于隐含空间知识融合的输电线路多金具检测方法。首先，为了挖掘输电线路金具间的隐含空间知识以协助模型进行检测，提出空间框设定模块和空间上下文提取模块进行空间框的设定以及空间上下文信息的提取。然后，设计空间上下文记忆模块对空间上下文信息进行筛选和记忆，并由此辅助多金具检测模型的定位。最后，改进模型后处理部分以进一步缓解金具密集遮挡带来的低检测精度问题。实验结果表明，该模型对多类金具的检测有提升效果，尤其对于小目标金具和密集遮挡金具的提升尤为显著。且相比于基线模型，在整体的AP⁵⁰评价指标和更严格的AP⁷⁵评价指标上分别提高了3.5%和5.7%。这为后续金具检测的落地应用和进一步的故障诊断奠定了基础。

关键词: 输电线路, 金具检测, 深度学习, 隐含空间知识, 空间上下文信息

Abstract:

To address the challenge of detecting tiny-size and dense occlusion objects in the task of multi-fitting detection for transmission lines, a transmission line multi-fitting detection method based on implicit spatial knowledge fusion was proposed. First, in order to mine the implicit spatial knowledge among transmission line fittings and assist the model in detection, the spatial box setting module and the spatial context extraction module were proposed to set the spatial box and extract the spatial context information. Then, the spatial context memory module was designed to filter and remember the spatial context information to aid the positioning of multi-fitting detection model. Finally, the post-processing part of the model was enhanced to further mitigate the issue of low detection accuracy stemming from dense occlusion by fittings. The experimental results demonstrated the efficacy of the proposed model in enhancing the detection of various kinds of fittings, especially those of tiny size and dense occlusion. Compared with the baseline model, the AP⁵⁰ evaluation index and the more stringent AP⁷⁵ evaluation index were increased by 3.5% and 5.7%, respectively. It laid a foundation for the application of fitting detection and further fault diagnosis.

Key words: transmission line, fitting detection, deep learning, implicit spatial knowledge, spatial context information

中图分类号:

TP391

翟永杰, 郭聪彬, 王乾铭, 赵宽, 白云山, 张冀. 基于隐含空间知识融合的输电线路多金具检测方法[J]. 图学学报, 2023, 44(5): 918-927.

ZHAI Yong-jie, GUO Cong-bin, WANG Qian-ming, ZHAO Kuan, BAI Yun-shan, ZHANG Ji. Multi-fitting detection method for transmission lines based on implicit spatial knowledge fusion[J]. Journal of Graphics, 2023, 44(5): 918-927.

图/表 11

图1 金具组合结构示例图((a)提包式固定结构；(b)均压屏蔽结构；(c)绝缘子联接结构；(d)压缩型固定联接结构)

Fig. 1 Example diagrams of fitting combination structure ((a) Bag-type fixed structure; (b) Grading-shielded structure; (c) Insulator connection structure; (d) Compression-type fixed connection structure)

图2 Double-Head模型整体结构图

Fig. 2 The overall structure of Double-Head model

图3 本文模型整体网络框架图

Fig. 3 The overall network framework of the proposed model

图4 空间框设定过程图

Fig. 4 The process of spatial box setting

图5 门控循环单元结构图

Fig. 5 The structure of gated recurrent unit

图6 各类金具外观示意图

Fig. 6 Appearance schematic diagrams of various fittings

表1 模型结构消融实验结果(%)

Table 1 The ablation experimental result of the model structure (%)

基线模型	隐含空间知识融合	改进后处理	AP⁵⁰	AP⁷⁵
√	-	-	77.5	41.9
√	√	-	81.0	44.9
√	-	√	78.2	45.0
√	√	√	81.0	47.6

表2 空间框系数消融实验结果(%)

Table 2 The ablation experimental result of the spatial box coefficient (%)

α	AP^50-95	AP⁵⁰	AP⁷⁵	AR¹	AR¹⁰	AR¹⁰⁰
0.10	45.4	80.1	45.7	26.5	56.0	59.1
0.15	45.3	80.0	46.1	26.3	56.2	58.9
0.20	46.1	80.1	47.9	27.3	56.3	59.1
0.25	46.6	81.0	47.6	27.3	56.9	59.6
0.30	45.6	80.2	47.5	26.8	55.6	58.7

表3 检测模型性能比较(%)

Table 3 The comparison of detection model performances (%)

方法	AP^50-95	AP⁵⁰	AP⁷⁵	AP^S	AP^M	AP^L	AR¹	AR¹⁰	AR¹⁰⁰	AR^S	AR^M	AR^L	Time (s/im)
SSD	33.5	65.8	30.4	13.9	14.0	37.5	22.4	41.5	42.6	20.6	22.1	46.8	0.008
RetinaNet	35.5	66.7	33.6	9.1	25.7	38.6	22.5	46.8	48.1	15.4	32.4	51.8	0.050
YOLOF	43.5	77.1	43.4	14.1	27.1	47.6	26.5	51.1	51.7	14.0	34.0	56.4	0.033
Dynamic R-CNN	43.7	75.1	45.4	19.6	34.0	46.8	26.4	51.8	52.2	19.6	39.2	55.9	0.041
Libra R-CNN	45.3	80.1	45.7	14.1	32.3	48.3	27.6	53.9	54.9	17.1	39.0	58.4	0.042
基线模型	42.1	77.5	41.9	14.9	32.9	45.0	25.7	50.1	50.9	15.1	39.4	54.0	0.069
本文模型	46.6	81.0	47.6	24.1	37.1	49.7	27.3	56.9	59.6	24.4	46.8	63.1	0.190

图7 每类金具检测精度对比图((a)每类金具AP50指标对比；(b)每类金具AP75指标对比)

Fig. 7 The comparison of detection accuracy for each type fitting ((a) The comparison of AP50 index for each type fitting; (b) The comparison of AP75 index for each type fitting)

图8 检测结果可视化对比图((a)~(c)基线模型检测结果可视化；(d)~(f)本文模型检测结果可视化)

Fig. 8 The comparison of detection result visualization ((a)~(c) The visualization of the baseline model detection results; (d)~(f) The visualization of the proposed model detection results)

参考文献 24

[1]	李德海, 李丹丹. 基于视觉的输电线路自动监测技术综述[J]. 黑龙江电力, 2019, 41(6): 559-564.
	LI D H, LI D D. Overview of vision-based transmission line automatic monitoring technology[J]. Heilongjiang Electric Power, 2019, 41(6): 559-564. (in Chinese)
[2]	赵振兵, 蒋志钢, 李延旭, 等. 输电线路部件视觉缺陷检测综述[J]. 中国图象图形学报, 2021, 26(11): 2545-2560.
	ZHAO Z B, JIANG Z G, LI Y X, et al. Overview of visual defect detection of transmission line components[J]. Journal of Image and Graphics, 2021, 26(11): 2545-2560. (in Chinese)
[3]	NGUYEN V N, JENSSEN R, ROVERSO D. Automatic autonomous vision-based power line inspection: a review of current status and the potential role of deep learning[J]. International Journal of Electrical Power & Energy Systems, 2018, 99: 107-120. DOI URL
[4]	刘传洋, 吴一全. 基于深度学习的输电线路视觉检测方法研究进[J/OL]. 中国电机工程学报. [2022-12-06]. http://kns.cnki.net/kcms/detail/11.2107.TM.20220831.1053.002.html.
	LIU C Y, WU Y Q. Research progress on visual detection methods for transmission lines based on deep learning[J/OL]. Proceedings of the CSEE. [2022-12-06]. http://kns.cnki.net/kcms/detail/11.2107.TM.20220831.1053.002.html. (in Chinese)
[5]	黄郑, 王红星, 翟学锋, 等. 输电线路无人机自主巡检方法研究与应用[J]. 计算技术与自动化, 2021, 40(3): 157-161.
	HUANG Z, WANG H X, ZHAI X F, et al. Research and application of UAV autonomous inspection method for transmission line[J]. Computing Technology and Automation, 2021, 40(3): 157-161. (in Chinese)
[6]	金立军, 胡娟, 闫书佳. 基于图像的高压输电线间隔棒故障诊断方法[J]. 高电压技术, 2013, 39(5): 1040-1045.
	JIN L J, HU J, YAN S J. Method of spacer fault diagnose on transmission line based on image procession[J]. High Voltage Engineering, 2013, 39(5): 1040-1045. (in Chinese)
[7]	万林, 巫世晶, 谢福起, 等. 基于图像处理的输电线路耐张线夹监测系统[J]. 武汉大学学报: 工学版, 2020, 53(12): 1106-1111.
	WAN L, WU S J, XIE F Q, et al. Monitoring system of transmission line strain clamps based on image processing[J]. Engineering Journal of Wuhan University, 2020, 53(12): 1106-1111. (in Chinese)
[8]	王胜, 陈文, 匡小兵, 等. 一种基于多特征显著性融合的绝缘子区域检测与定位算法[J]. 计算机应用研究, 2020, 37(S2): 351-353.
	WANG S, CHEN W, KUANG X B, et al. Insulator area detection and localization algorithm based on multi-feature salient fusion[J]. Application Research of Computers, 2020, 37(S2): 351-353. (in Chinese)
[9]	戚银城, 江爱雪, 赵振兵, 等. 基于改进SSD模型的输电线路巡检图像金具检测方法[J]. 电测与仪表, 2019, 56(22): 7-12, 43.
	QI Y C, JIANG A X, ZHAO Z B, et al. Fittings detection method in patrol images of transmission line based on improved SSD[J]. Electrical Measurement & Instrumentation, 2019, 56(22): 7-12, 43. (in Chinese)
[10]	张永翔, 吴功平, 刘中云, 等. 基于YOLOv3网络的输电线路防震锤和线夹检测迁移学习[J]. 计算机应用, 2020, 40(S2): 188-194.
	ZHANG Y X, WU G P, LIU Z Y, et al. Transfer learning of transmission line damper and clamp detection based on YOLOv3 network[J]. Journal of Computer Applications, 2020, 40(S2): 188-194. (in Chinese)
[11]	李鑫, 刘帅男, 杨桢, 等. 基于改进Cascade R-CNN的输电线路多目标检测[J]. 电子测量与仪器学报, 2021, 35(10): 24-32.
	LI X, LIU S N, YANG Z, et al. Multi-target detection of transmission lines based on improved cascade R-CNN[J]. Journal of Electronic Measurement and Instrumentation, 2021, 35(10): 24-32. (in Chinese)
[12]	易继禹, 陈慈发, 龚国强. 基于改进Faster RCNN的输电线路航拍绝缘子检测[J]. 计算机工程, 2021, 47(6): 292-298, 304. DOI
	YI J Y, CHEN C F, GONG G Q. Aerial insulator detection of transmission line based on improved faster RCNN[J]. Computer Engineering, 2021, 47(6): 292-298, 304. (in Chinese) DOI
[13]	WU Y, CHEN Y P, YUAN L, et al. Rethinking classification and localization for object detection[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 10186-10195.
[14]	BODLA N, SINGH B, CHELLAPPA R, et al. Soft-NMS--improving object detection with one line of code[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 5561-5569.
[15]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[16]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[17]	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 2117-2125.
[18]	HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2961-2969.
[19]	CHO K, VAN MERRIËNBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[EB/OL]. [2023-8-16]. https://arxiv.org/abs/1406.1078.
[20]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// 2016 European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[21]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2980-2988.
[22]	CHEN Q, WANG Y M, YANG T, et al. You only look one-level feature[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13039-13048.
[23]	ZHANG H K, CHANG H, MA B P, et al. Dynamic R-CNN: towards high quality object detection via dynamic training[C]// 2020 European Conference on Computer Vision. Cham: Springer, 2020: 260-275.
[24]	PANG J M, CHEN K, SHI J P, et al. Libra R-CNN: towards balanced learning for object detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 821-830.

基于隐含空间知识融合的输电线路多金具检测方法

Multi-fitting detection method for transmission lines based on implicit spatial knowledge fusion

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 24

相关文章 15

编辑推荐

Metrics

本文评价

[1]	周锐闯, 田瑾, 闫丰亭, 朱天晓, 张玉金 . 融合外部注意力和图卷积的点云分类模型 [J]. 图学学报, 2023, 44(6): 1162-1172.
[2]	王吉, 王森, 蒋智文, 谢志峰, 李梦甜. 基于深度条件扩散模型的零样本文本驱动虚拟人生成方法 [J]. 图学学报, 2023, 44(6): 1218-1226.
[3]	阎光伟, 刘润泽, 焦润海, 何慧. 基于改进Cascade RCNN的输电线路防振锤脱落检测方法[J]. 图学学报, 2023, 44(5): 849-860.
[4]	杨陈成, 董秀成, 侯兵, 张党成, 向贤明, 冯琪茗. 基于参考的Transformer纹理迁移深度图像超分辨率重建[J]. 图学学报, 2023, 44(5): 861-867.
[5]	党宏社, 许怀彪, 张选德. 融合结构信息的深度学习立体匹配算法[J]. 图学学报, 2023, 44(5): 899-906.
[6]	杨红菊, 高敏, 张常有, 薄文, 武文佳, 曹付元. 一种面向图像修复的局部优化生成模型[J]. 图学学报, 2023, 44(5): 955-965.
[7]	毕春艳, 刘越. 基于深度学习的视频人体动作识别综述[J]. 图学学报, 2023, 44(4): 625-639.
[8]	郝帅, 赵新生, 马旭, 张旭, 何田, 侯李祥. 基于TR-YOLOv5的输电线路多类缺陷目标检测方法[J]. 图学学报, 2023, 44(4): 667-676.
[9]	曹义亲, 周一纬, 徐露. 基于E-YOLOX的实时金属表面缺陷检测算法[J]. 图学学报, 2023, 44(4): 677-690.
[10]	邵俊棋, 钱文华, 徐启豪. 基于条件残差生成对抗网络的风景图生成[J]. 图学学报, 2023, 44(4): 710-717.
[11]	余伟群, 刘佳涛, 张亚萍. 融合注意力的拉普拉斯金字塔单目深度估计[J]. 图学学报, 2023, 44(4): 728-738.
[12]	郭印宏, 王立春, 李爽. 基于重复性和特异性约束的图像特征匹配[J]. 图学学报, 2023, 44(4): 739-746.
[13]	毛爱坤, 刘昕明, 陈文壮, 宋绍楼. 改进YOLOv5算法的变电站仪表目标检测方法[J]. 图学学报, 2023, 44(3): 448-455.
[14]	王佳婧, 王晨, 朱媛媛, 王笑梅. 基于民国纸币的图元素匹配检索[J]. 图学学报, 2023, 44(3): 492-501.
[15]	杨柳, 吴晓群. 基于深度学习的三维形状补全研究综述[J]. 图学学报, 2023, 44(2): 201-215.