Multi-fitting detection for transmission lines based on a cascade query-position relationship method

doi:10.11996/JG.j.2095-302X.2025020288

Abstract

Abstract:

To address the challenges posed by the small-size and dense occlusion of fittings in aerial images of transmission lines, a multi-fitting detection method for transmission lines based on a cascade query-position relationship (CQPR) was proposed. Firstly, a cascade sparse query module was proposed to query the precise location of small-size objects in large-scale feature maps using the rough location of small-size objects on small-scale feature maps, thereby improving the accuracy of small-size fitting detection. Then, a positional feature relationship module (PRM) was proposed to optimize the detection results at the occlusion using the positional relationships between different fittings in the image to establish the positional PRM. This enriched the features of occluded areas and optimized fitting detection performance under conditions of dense occlusion. Experimental results on multiple baselines demonstrated that when the proposed CQPR for mult-fitting detection in transmission lines was applied to the baseline, achieving the accuracy of Faster R-CNN, Cascade R-CNN, Libra R-CNN, and Dynamic R-CNN at 82.9%, 82.4%, 83.7%, and 77.3%, respectively. These results surpassed those of other state-of-the-art object detection models, particularly in the accuracy of detection for small-size fittings and fittings with occlusion. The inference speed was also improved, achieving a balance between localization accuracy and real-time detection.

Key words: transmission lines, fitting, deep learning, object detection, small-size object, dense occlusion

CLC Number:

TP391
TM75

ZHAI Yongjie, WANG Luyao, ZHAO Xiaoyu, HU Zhedong, WANG Qianming, WANG Yaru. Multi-fitting detection for transmission lines based on a cascade query-position relationship method[J]. Journal of Graphics, 2025, 46(2): 288-299.

Figures/Tables 14

References 30

[1]	中能传媒研究院. 中国能源大数据报告(2023): 第一章能源发展综述[EB/OL]. (2023-06-20) [2024-04-06]. https://news.bjx.com.cn/html/20230620/1313822.shtml
	China Energy Media Academy. China energy big data report (2023)[EB/OL]. (2023-06-20) [2024-04-06]. https://news.bjx.com.cn/html/20230620/1313822.shtml (in Chinese).
[2]	李德海, 李丹丹. 基于视觉的输电线路自动监测技术综述[J]. 黑龙江电力, 2019, 41(6): 559-564.
	LI D H, LI D D. Overview of vision-based transmission line automatic monitoring technology[J]. Heilongjiang Electric Power, 2019, 41(6): 559-564 (in Chinese).
[3]	ZHAO Z B, LIU N. The recognition and localization of insulators adopting SURF and IFS based on correlation coefficient[J]. Optik, 2014, 125(20): 6049-6052.
[4]	陈文师. 高压输电线路的视觉巡检方法研究[J]. 现代制造, 2017(18): 152-153.
	CHEN W S. Research on visual inspection method of high voltage transmission line[J]. Mechanical and Electrical Information, 2017(18): 152-153 (in Chinese).
[5]	高文婷, 刘越. 面向移动增强现实的实时深度学习目标检测方法综述[J]. 图学学报, 2021, 42(4): 525-534.
	GAO W T, LIU Y. Review of real-time deep learning-based object detection for mobile augmented reality[J]. Journal of Graphics, 2021, 42(4): 525-534 (in Chinese).
[6]	孙宗康, 饶睦敏, 曹裕灵, 等. 基于小样本不均衡数据的供水管道泄漏智能检测算法[J]. 图学学报, 2022, 43(5): 825-831.
	SUN Z K, RAO M M, CAO Y L, et al. Water supply pipeline leakage intelligent detection algorithm based on small and unbalanced data[J]. Journal of Graphics, 2022, 43(5): 825-831 (in Chinese).
[7]	赵振兵, 吕雪纯, 王帆帆, 等. 基于改进YOLOx-S的输电线路上金具检测方法[J]. 无线电工程, 2023, 53(11): 2664-2672.
	ZHAO Z B, LYU X C, WANG F F, et al. Fitting detection on transmission line based on improved YOLOx-S[J]. Radio Engineering, 2023, 53(11): 2664-2672 (in Chinese).
[8]	赵振兵, 吕雪纯, 苗思雨, 等. 基于共有特征评分的解耦知识蒸馏输电线路金具检测方法[EB/OL]. (2023-11-13) [2024- 04-03]. https://doi.org/10.13336/j.1003-6520.hve.20231541.
	ZHAO Z B, LÓ X C, MIAO S Y, et al. Decoupled knowledge distillation transmission line fittings inspection method based on shared feature scoring[EB/OL]. (2023-11-13)[2024-04-03]. https://doi.org/10.13336/j.1003-6520.hve.20231541 (in Chinese).
[9]	张涛, 高绅, 陈万培, 等. 基于深度学习的间隔棒故障识别方法[J]. 无线电工程, 2021, 51(6): 502-506.
	ZHANG T, GAO S, CHEN W P, et al. Spacer bar fault detection based on deep learning[J]. Radio Engineering, 2021, 51(6): 502-506 (in Chinese).
[10]	HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 4700-4708.
[11]	VIEIRA-E-SILVA A L B, DE CASTRO FELIX H, DE MENEZES CHAVES T, et al. STN PLAD: a dataset for multi-size power line assets detection in high-resolution UAV images[C]// The 34th SIBGRAPI Conference on Graphics, Patterns and Images. New York: IEEE Press, 2021: 215-222.
[12]	戚银城, 赵席彬, 耿劭锋, 等. 基于遮挡关系推理的输电线路图像金具检测[J]. 智能系统学报, 2022, 17(6): 1154-1162.
	QI Y C, ZHAO X B, GENG S F, et al. Fittings detection in transmission line images with occlusion relation inference[J]. CAAI Transactions on Intelligent Systems, 2022, 17(6): 1154-1162 (in Chinese).
[13]	YANG C H Y, HUANG Z H, WANG N Y. QueryDet: cascaded sparse query for accelerating high-resolution small object detection[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13658-13667.
[14]	GRAHAM B, VAN DER MAATEN L. Submanifold sparse convolutional networks[EB/OL]. [2024-05-15]. https://arxiv.org/abs/1706.01307.
[15]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 2980-2988.
[16]	HU H, GU J Y, ZHANG Z, et al. Relation networks for object detection[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 3588-3597.
[17]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// The 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.
[18]	PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library[EB/OL]. [2024-05-15]. https://dblp.uni-trier.de/db/conf/nips/nips2019.html#PaszkeGMLBCKLGA19.
[19]	CHEN K, WANG J Q, PANG J M, et al. MMDetection: open MMLab detection toolbox and benchmark[EB/OL]. [2024-05-15]. https://arxiv.org/abs/1906.07155.
[20]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-Time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[21]	CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 6154-6162.
[22]	PANG J M, CHEN K, SHI J P, et al. Libra R-CNN: towards balanced learning for object detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 821-830.
[23]	ZHANG H K, CHANG H, MA B P, et al. Dynamic R-CNN: towards high quality object detection via dynamic training[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 260-275.
[24]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]// The 13th European Conference on Computer Vision. Cham: Springer, 2014: 740-755.
[25]	WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 7464-7475.
[26]	JOCHER G, CHAURASIA A, QIU J. Ultralytics YOLO (Version 8.0.0)[EB/OL]. [2024-07-06]. https://github.com/ultralytics/ultralytics.
[27]	LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]// The 14th European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[28]	KONG T, SUN F C, LIU H P, et al. FoveaBox: beyound anchor-based object detection[J]. IEEE Transactions on Image Processing, 2020, 29: 7389-7398.
[29]	FENG C, ZHONG Y, GAO Y, et al. Tood: Task-aligned one-stage object detection[C]// 2021 IEEE/CVF International Conference on Computer Vision. IEEE Computer Society, 2021: 3490-3499.
[30]	CHEN Z H, YANG C H Y, LI Q F, et al. Disentangle your dense object detector[C]// The 29th ACM International Conference on Multimedia. New York: ACM, 2021: 4939-4948.

模型	方法	AP_50-95	AP₅₀	AP₇₅	AP_S	AP_M	AP_L	AR₁₀₀	FPS
Faster R-CNN^[20]	Baseline	39.4	78.9	35.7	14.7	26.9	41.5	48.1	16.4
Faster R-CNN^[20]	CQPR Faster	42.9	82.9	39.9	18.8	29.4	44.8	50.6	25.0
Cascade R-CNN^[21]	Baseline	41.5	79.4	39.3	18.0	29.5	43.7	51.2	14.7
Cascade R-CNN^[21]	CQPR Cascade	43.9	82.4	43.1	20.2	34.7	45.8	52.3	10.8
Libra R-CNN^[22]	Baseline	40.5	80.3	36.4	18.3	28.4	42.4	49.6	10.7
Libra R-CNN^[22]	CQPR Libra	43.0	83.7	40.5	18.5	35.7	44.8	51.0	24.0
Dynamic R-CNN^[23]	Baseline	38.7	75.1	36.2	12.8	29.1	40.7	46.6	25.9
Dynamic R-CNN^[23]	CQPR Dynamic	41.7	77.3	40.0	19.4	35.4	43.3	49.4	24.7

模型	方法	AP_50-95	AP₅₀	AP₇₅	AP_S	AP_M	AP_L	AR₁₀₀	FPS
Faster R-CNN^[20]	Baseline	39.4	78.9	35.7	14.7	26.9	41.5	48.1	16.4
Faster R-CNN^[20]	CQPR Faster	42.9	82.9	39.9	18.8	29.4	44.8	50.6	25.0
Cascade R-CNN^[21]	Baseline	41.5	79.4	39.3	18.0	29.5	43.7	51.2	14.7
Cascade R-CNN^[21]	CQPR Cascade	43.9	82.4	43.1	20.2	34.7	45.8	52.3	10.8
Libra R-CNN^[22]	Baseline	40.5	80.3	36.4	18.3	28.4	42.4	49.6	10.7
Libra R-CNN^[22]	CQPR Libra	43.0	83.7	40.5	18.5	35.7	44.8	51.0	24.0
Dynamic R-CNN^[23]	Baseline	38.7	75.1	36.2	12.8	29.1	40.7	46.6	25.9
Dynamic R-CNN^[23]	CQPR Dynamic	41.7	77.3	40.0	19.4	35.4	43.3	49.4	24.7

模型	方法	PT	UT	SP	WT	YP	HB	CT	BT	AB	PG
Faster R-CNN^[20]	Baseline	84.5	76.7	85.3	76.2	77.7	76.2	56.5	94.0	83.9	78.2
Faster R-CNN^[20]	CQPR Faster	93.9	77.5	89.4	82.5	81.0	79.9	62.5	94.2	85.8	81.9
Cascade R-CNN^[21]	Baseline	91.4	76.6	88.1	70.5	76.3	77.2	59.5	94.7	84.7	75.4
Cascade R-CNN^[21]	CQPR Cascade	91.6	78.5	88.0	79.1	78.9	79.1	65.3	93.2	85.2	85.0
Libra R-CNN^[22]	Baseline	89.1	78.9	87.5	77.1	77.5	78.4	62,1	95.5	84.6	72.0
Libra R-CNN^[22]	CQPR Libra	93.7	80.5	89.4	82.5	79.8	80.1	64.5	94.7	85.2	86.3
Dynamic R-CNN^[23]	Baseline	91.3	74.1	79.9	58.8	67.4	75.7	52.6	93.9	83.8	72.8
Dynamic R-CNN^[23]	CQPR Dynamic	85.3	76.4	82.3	59.5	73.9	77.2	56.0	93.3	84.1	85.4

模型	方法	PT	UT	SP	WT	YP	HB	CT	BT	AB	PG
Faster R-CNN^[20]	Baseline	84.5	76.7	85.3	76.2	77.7	76.2	56.5	94.0	83.9	78.2
Faster R-CNN^[20]	CQPR Faster	93.9	77.5	89.4	82.5	81.0	79.9	62.5	94.2	85.8	81.9
Cascade R-CNN^[21]	Baseline	91.4	76.6	88.1	70.5	76.3	77.2	59.5	94.7	84.7	75.4
Cascade R-CNN^[21]	CQPR Cascade	91.6	78.5	88.0	79.1	78.9	79.1	65.3	93.2	85.2	85.0
Libra R-CNN^[22]	Baseline	89.1	78.9	87.5	77.1	77.5	78.4	62,1	95.5	84.6	72.0
Libra R-CNN^[22]	CQPR Libra	93.7	80.5	89.4	82.5	79.8	80.1	64.5	94.7	85.2	86.3
Dynamic R-CNN^[23]	Baseline	91.3	74.1	79.9	58.8	67.4	75.7	52.6	93.9	83.8	72.8
Dynamic R-CNN^[23]	CQPR Dynamic	85.3	76.4	82.3	59.5	73.9	77.2	56.0	93.3	84.1	85.4

模型	PRM	CSQN	AP	AP₅₀	AP₇₅	AP_S	AP_M	AP_L	FPS
Faster R-CNN			39.4	78.9	35.7	14.7	26.9	41.5	16.4
	√		42.3	82.0	39.4	16.2	32.6	44.2	15.1
		√	42.2	82.0	39.2	19.5	32.2	44.1	27.4
	√	√	42.9	82.9	39.9	18.8	29.4	44.8	25.0
Cascade R-CNN			41.5	79.4	39.3	18.0	29.5	43.7	14.7
	√		43.0	80.5	42.2	18.7	31.7	45.0	7.1
		√	44.1	81.9	42.5	21.0	30.5	46.1	23.0
	√	√	43.9	82.4	43.1	20.2	34.7	45.8	10.8
Libra R-CNN			40.5	80.3	36.4	18.3	28.4	42.4	10.7
	√		43.4	81.7	40.7	18.8	37.5	45.2	14.8
		√	43.2	81.9	38.5	20.8	32.5	45.2	25.9
	√	√	43.0	83.7	40.5	18.5	35.7	44.8	24.0
Dynamic R-CNN			38.7	75.1	36.2	12.8	29.1	40.7	25.9
	√		40.8	75.6	39.6	18.4	36.3	42.5	15.8
		√	40.5	75.8	39.3	20.1	31.2	42.5	26.9
	√	√	41.7	77.3	40.0	19.4	35.4	43.3	24.7