Detection method of dropped anti-vibration hammer for transmission line based on improved Cascade RCNN

doi:10.11996/JG.j.2095-302X.2023050849

Abstract

Abstract:

During the inspection of transmission lines using Unmanned Aerial Vehicle (UAV), there are many dropped anti-vibration hammers that become obstructed by wires or are shot from a distance. This challenge leads to the occlusion of target features and low resolution. In addition, the close proximity of a number of hammers due to sliding poses challenges to the accuracy of target identification. To address the above problems, a deep neural network based on an improved Cascade RCNN was proposed to identify the dropped anti-vibration hammers. The proposed network mainly achieved improvements from the following four aspects. First of all, a contrastive learning network was designed to compare the features of positive and negative samples with real samples. By utilizing a contrastive loss function during network training, the network became more attentive to the blocked dropped anti-vibration hammers and enhanced its feature extraction ability. Secondly, the classifier was enhanced. The selection of interested regions with better regression performance in the cascade structure was filtered and input directly into the final classification regression queue. This improved the classification performance of the classifier, thereby enhancing the classification scores of the detected targets. Thirdly, a parallel attention mechanism module was designed to integrate the extracted features from the network, increasing the weights of key features and directing the network’s attention to more critical features in the image. In addition, in the process of feature fusion of the feature pyramid, the bilinear interpolation method was replaced with deconvolution to enhance the feature restoration capability. The experimental results demonstrated that the improved model achieved a recall rate of 97.5%, precision of 91.0%, and average precision of 92.0%, an improvement of 6.9%, 28.4%, and 8.0%, respectively, compared with the baseline model.

Key words: transmission line, dropped anti-vibration hammer, Cascade RCNN, contrastive learning network, parallel attention module, classifier enhancement, sample similarity

CLC Number:

TP391.4

YAN Guang-wei, LIU Run-ze, JIAO Run-hai, HE Hui. Detection method of dropped anti-vibration hammer for transmission line based on improved Cascade RCNN[J]. Journal of Graphics, 2023, 44(5): 849-860.

Figures/Tables 25

Fig. 1 Cfce-Pa-Net network architecture

Fig. 2 Contrastive learning network structure

Fig. 3 Different thresholds α test results under ((a) α=0; (b) α=0.5)

Fig. 4 ROI position regression change statistical chart

Fig. 5 Classifier enhancement structure diagram

Fig. 6 Design diagram of parallel attention module

Table 1 Confusion matrix

真实情况	预测结果
真实情况	正例	反例
正例	TP (真正例)	FN (假负例)
反例	FP (假正例)	TN (真负例)

Fig. 7 The detection results of Cfce-Pa-Net ((a) For occluded object detection; (b) Long-range target detection; (c) Sliding target detection)

Table 2 Comparison between the model proposed in this paper and other models

Model	Backbone	Contrastive loss	Classifier enhancement	Parallel attention	AP (%)	Time
Model	Backbone	Contrastive loss	Classifier enhancement	Parallel attention	AP (%)	Train (h)	Test (s)
SSD	VGG-16	×	×	×	75.3	5.1	0.160
YOLOv4^[9]	Darknet-53	×	×	×	80.7	6.4	0.165
YOLOv5^[10]	CSPDarknet53+Focus	×	×	×	86.3	7.5	0.113
Faster RCNN^[7]	ResNet-101	×	×	×	83.0	13.7	0.165
Cascade RCNN^[8]	ResNet-101	×	×	×	84.0	10.0	0.160
Cfce-Pa-Net1	ResNet-101	√	×	×	89.1	15.3	0.169
Cfce-Pa-Net2	ResNet-101	√	√	×	90.3	17.3	0.169
Cfce-Pa-Net3	ResNet-101	√	√	√	92.0	20.1	0.171
Cfce-Pa-Net3	ResNet-50	√	√	√	91.8	19.3	0.168

Table 3 Comparing the impact of different similarity calculation methods on networks (%)

相似度计算方式	AP	Precision	Recall
欧式距离	89.1	78.7	95.0
余弦相似度	88.2	48.2	94.9
点积相似度	88.3	69.1	93.5

Fig. 8 Characteristic thermal map ((a) Original image; (b) The feature heat map extracted by the network before improvement; (c) Improved network extracted feature thermal maps)

Fig. 9 Changes in Euclidean distance distribution between positive and real samples

Fig. 10 Classification regression loss change chart ((a) Classification loss; (b) Regression loss)

Table 4 Different thresholds β experimental results

方法	阈值β	AP (%)	Precision (%)	Recall (%)
不加入分支	-	89.1	78.7	95.0
加入分支	0.80	80.7	66.0	88.7
	0.85	84.1	62.9	90.1
	0.90	87.0	57.4	93.7
	0.95	90.3	89.0	95.9
	0.96	90.1	85.8	96.3
	0.97	90.1	80.8	96.2

Fig. 11 Comparison of results before and after classifier enhancement ((a) Improved; (b) Before improvement)

Table 5 Comparison table of ablation experiment effects (%)

方法	Recall	Precision	AP
Cascade RCNN^[8]	95.9	89.0	90.3
Cascade RCNN^[8]+通道注意力	95.7	80.8	90.4
Cascade RCNN^[8]+空间注意力	95.9	85.8	90.6
Cascade RCNN^[8]+并行注意力	97.5	91.0	92.0

Fig. 12 Visualization of thermal maps ((a) Original images; (b) Thermal diagram before improvement; (c) Improved thermal diagram)

Table 6 Comparative experiment of deconvolution and bilinear interpolation methods (%)

上采样方式	Recall	Precision	AP
反卷积	97.5	91.0	92.0
双线性插值	97.0	88.6	91.3

Table 7 Baseline network cross validation experimental results (%)

测试集	Recall	Precision	AP
1	90.1	62.9	84.1
2	90.8	60.1	83.7
3	90.2	66.2	84.3
4	92.8	57.5	83.9
5	89.0	66.3	84.2

Table 8 Contrastive learning network cross validation experimental results (%)

测试集	Recall	Precision	AP
1	95.6	75.4	88.9
2	96.2	82.7	89.2
3	92.3	80.2	88.8
4	93.2	85.6	89.2
5	97.6	69.5	89.3

Table 9 Experimental results of classifier enhancement cross validation (%)

测试集	Recall	Precision	AP
1	95.1	89.2	90.3
2	96.2	89.9	90.4
3	96.3	85.8	90.1
4	96.2	90.0	90.5
5	95.6	90.0	90.1

Table 10 Experimental results of parallel attention cross validation (%)

测试集	Recall	Precision	AP
1	95.6	90.4	91.9
2	98.6	88.4	92.1
3	98.1	92.0	91.8
4	97.0	92.8	92.1
5	98.3	91.5	91.9

Fig. 13 Insulator self explosion defect ((a), (b) Insulator self explosion defects from different perspectives)

Table 11 Experimental results of insulator self explosion dataset (%)

方法	AP	Precision	Recall
Cascade RCNN^[8]	85.4	50.9	88.6
Cfce-Pa-Net1	87.7	59.7	91.9
Cfce-Pa-Net2	89.2	81.2	91.3
Cfce-Pa-Net3	89.6	81.4	91.1

Fig. 14 Comparison of detection results before and after model improvement (a) Before improvement; (b) After improvement)

References 25

[1]	东辉, 陈鑫凯, 孙浩, 等. 基于改进YOLOv4和图像处理的蔬菜田杂草检测[J]. 图学学报, 2022, 43(4): 559-569.
	DONG H, CHEN X K, SUN H, et al. Weed detection in vegetable field based on improved YOLOv4 and image processing[J]. Journal of Graphics, 2022, 43(4): 559-569. (in Chinese)
[2]	张运波, 易鹏飞, 周东生, 等. 深度可分离卷积和标准卷积相结合的高效行人检测器[J]. 图学学报, 2022, 43(2): 230-238.
	ZHANG Y B, YI P F, ZHOU D S, et al. Efficient pedestrian detector combining depthwise separable convolution and standard convolution[J]. Journal of Graphics, 2022, 43(2): 230-238. (in Chinese)
[3]	胡俊, 顾晶晶, 王秋红. 基于遥感图像的多模态小目标检测[J]. 图学学报, 2022, 43(2): 197-204.
	HU J, GU J J, WANG Q H. Multimodal small target detection based on remote sensing image[J]. Journal of Graphics, 2022, 43(2): 197-204. (in Chinese) DOI
[4]	陈晓娟, 吴英石, 赵亮. 基于随机Hough变换的OPGW防震锤识别[J]. 黑龙江电力, 2010, 32(1): 1-2, 5.
	CHEN X J, WU Y S, ZHAO L. Identification of OPGW vibration damper based on random Hough transformation[J]. Heilongjiang Electric Power, 2010, 32(1): 1-2, 5. (in Chinese)
[5]	谭磊, 王耀南, 沈春生. 输电线路除冰机器人障碍视觉检测识别算法[J]. 仪器仪表学报, 2011, 32(11): 2564-2571.
	TAN L, WANG Y N, SHEN C S. Visual detection and recognition algorithm for obstacles of power transmission line deicing robot[J]. Journal of Instrumentation, 2011, 32(11): 2564-2571. (in Chinese)
[6]	金立军, 闫书佳, 刘源. 基于类Haar特征与级联AdaBoost算法的防震锤识别[J]. 系统仿真学报, 2012, 24(9): 1806-1809.
	JIN L J, YAN S J, LIU Y. Vibration damper recognition based on haar-like features and cascade AdaBoost classifier[J]. Journal of System Simulation, 2012, 24(9): 1806-1809. (in Chinese)
[7]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. DOI PMID
[8]	CAI Z W, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 6154-6162.
[9]	BOCHKOVSKIY A, WANG C Y, LIAO H-Y M. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2022-11-18]. https://arxiv.org/abs/2004.10934.
[10]	ZHU X K, LYU S C, WANG X, et al. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops. New York: IEEE Press, 2021: 2778-2788.
[11]	张家盛, 梁进兴. 基于深度学习的无人机巡检架空输电线路金具锈蚀缺陷检测方法[J]. 湖南电力, 2022, 42(5): 75-78. DOI
	ZHANG J S, LIANG J X. Detection method of metal fitting rust defects for overhead transmission lines based on UAV patrol of deep learning[J]. Hunan Electric Power, 2022, 42(5): 75-78. (in Chinese) DOI
[12]	许扬, 凌德泉, 严锋, 等. 基于EfficientDet的输电线路破损防振锤检测[J]. 电子设计工程, 2022, 30(9): 139-143.
	XU Y, LING D Q, YAN F, et al. EfficientDet-based detection of damaged vibration damper in transmission line[J]. Electronic Design Engineering, 2022, 30(9): 139-143. (in Chinese)
[13]	安金鹏, 王晓春. 基于深度学习和视觉语义关系的防振锤滑移目标检测[J]. 智能计算机与应用, 2022, 12(9): 183-188.
	AN J P, WANG X C. Target detection of vibration damper movement based on deep learning and visual semantic relations[J]. Intelligent Computer and Applications, 2022, 12(9): 183-188. (in Chinese)
[14]	贾雁飞, 陈广大, 杨淼, 等. 基于深度学习的高压输电线路防振锤检测[J]. 机床与液压, 2022, 50(13): 21-25. DOI
	JIA Y F, CHEN G D, YANG M, et al. Damper detection of high voltage transmission line based on deep learning[J]. Machine Tool & Hydraulics, 2022, 50(13): 21-25. (in Chinese)
[15]	CAI D L, ZHANG Z Y, HE G F. A study on the effectiveness of detection of unbalanced datasets based on faster R-CNN[C]// The 12th International Conference on Power, Energy and Electrical Engineering. New York: IEEE Press, 2022: 321-326.
[16]	LI Y F, DAI L G, NI H X, et al. Focus on local: transmission line defect detection via feature refinement[J]. Signal, Image and Video Processing, 2023, 17: 31-37. DOI
[17]	张永翔, 吴功平, 刘中云, 等. 基于YOLOv3网络的输电线路防震锤和线夹检测迁移学习[J]. 计算机应用, 2020, 40(S2): 188-194.
	ZHANG Y X, WU G P, LIU Z Y, et al. Transfer learning of transmission line damper and clamp detection based on YOLOv3 network[J]. Journal of Computer Applications, 2020, 40(S2): 188-194. (in Chinese)
[18]	刘又维, 樊绍胜, 唐立军, 等. 基于关键点的输电线路防振锤滑移视觉检测[J]. 激光与光电子学进展, 2020, 57(20): 279-285.
	LIU Y W, FAN S S, TANG L J, et al. Visual detection of transmission line anti-vibration hammer slip based on key points[J]. Laser & Optoelectronics Progress, 2020, 57(20): 279-285. (in Chinese)
[19]	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 1314-1324.
[20]	罗玉鹤, 庞红旗, 高飞翎, 等. 电力巡检图像中防振锤的区域全卷积网络识别方法[J]. 福州大学学报: 自然科学版, 2020, 48(6): 714-719.
	LUO Y H, PANG H Q, GAO F L, et al. Identification method of region based fully convolution networks for damper in power inspection image[J]. Journal of Fuzhou University: Natural Science Edition, 2020, 48(6): 714-719. (in Chinese)
[21]	缪希仁, 刘欣宇, 江灏, 等. 一种基于深度学习和距离约束的航拍图像防震锤滑移故障诊断方法: CN108596883A[P]. 2021-07-13.
	MIAO X R, LIU X Y, JIANG H, et al. Deep learning and distance constraint-based aerial image vibration damper slippage fault diagnosis method: CN108596883A[P]. 2021-07-13. (in Chinese)
[22]	NI H X, WANG M Z, ZHAO L Y. An improved Faster R-CNN for defect recognition of key components of transmission line[J]. Mathematical Biosciences and Engineering, 2021, 18(4): 4679-4695. DOI PMID
[23]	ZHANG Z, JIANG W, YANG J J. An improved quantization algorithm for electric power inspection[C]// The 9th International Electrical Engineering Congress. New York: IEEE Press, 2021: 397-400.
[24]	SUN X D, HAO H, LIU Y, et al. Research on the application of YOLOv4 in power inspection[J]. IOP Conference Series: Earth and Environmental Science, 2021, 693(1): 12038. DOI
[25]	CHEN T, KORNBLITH S, SWERSKY K, et al. Big self-supervised models are strong semi-supervised learners[EB/OL]. [2022-12-29]. ttps://arxiv.org/abs/2006.10029v2.