Journal of Graphics ›› 2023, Vol. 44 ›› Issue (3): 438-447.DOI: 10.11996/JG.j.2095-302X.2023030438
Previous Articles Next Articles
LI Gang1,2(), ZHANG Yun-tao1, WANG Wen-kai1, ZHANG Dong-yang1()
Received:
2022-08-12
Accepted:
2022-12-28
Online:
2023-06-30
Published:
2023-06-30
Contact:
ZHANG Dong-yang (1981-), engineer, master. His main research interest covers computer vision. E-mail:zhdy@ncepu.edu.cn
About author:
LI Gang (1980-), associate professor, Ph.D. His main research interests cover prognostics and health management, etc. E-mail:ququ_er2003@126.com
Supported by:
CLC Number:
LI Gang, ZHANG Yun-tao, WANG Wen-kai, ZHANG Dong-yang. Defect detection method of transmission line bolts based on DETR and prior knowledge fusion[J]. Journal of Graphics, 2023, 44(3): 438-447.
Add to citation manager EndNote|Ris|BibTeX
URL: http://www.txxb.com.cn/EN/10.11996/JG.j.2095-302X.2023030438
迭代 次数 | 学习率 | |||
---|---|---|---|---|
0.01 | 0.001 | 0.000 5 | 0.000 1 | |
1 000 | 46.73 | 45.39 | 45.23 | 44.85 |
2 000 | 48.65 | 45.63 | 48.14 | 46.32 |
4 000 | 49.11 | 49.17 | 49.96 | 47.64 |
6 000 | 49.30 | 48.96 | 49.96 | 48.59 |
Table 1 The average detection rate of the model under different learning rates and iterations (%)
迭代 次数 | 学习率 | |||
---|---|---|---|---|
0.01 | 0.001 | 0.000 5 | 0.000 1 | |
1 000 | 46.73 | 45.39 | 45.23 | 44.85 |
2 000 | 48.65 | 45.63 | 48.14 | 46.32 |
4 000 | 49.11 | 49.17 | 49.96 | 47.64 |
6 000 | 49.30 | 48.96 | 49.96 | 48.59 |
Mini-batch | AP | ||||
---|---|---|---|---|---|
正常螺栓 | 缺销钉 | 缺螺母 | 缺垫片 | AP_0.5 | |
8 | 71.6 | 59.6 | 53.6 | 55.6 | 60.1 |
16 | 75.0 | 61.4 | 54.1 | 58.7 | 62.3 |
32 | 74.7 | 64.0 | 54.5 | 60.4 | 63.4 |
64 | 78.6 | 57.6 | 54.7 | 61.1 | 63.0 |
128 | 83.3 | 51.1 | 54.2 | 61.8 | 62.6 |
Table 2 The average precision and average precision of each category corresponding to different Mini-batch parameters (%)
Mini-batch | AP | ||||
---|---|---|---|---|---|
正常螺栓 | 缺销钉 | 缺螺母 | 缺垫片 | AP_0.5 | |
8 | 71.6 | 59.6 | 53.6 | 55.6 | 60.1 |
16 | 75.0 | 61.4 | 54.1 | 58.7 | 62.3 |
32 | 74.7 | 64.0 | 54.5 | 60.4 | 63.4 |
64 | 78.6 | 57.6 | 54.7 | 61.1 | 63.0 |
128 | 83.3 | 51.1 | 54.2 | 61.8 | 62.6 |
Encoder参数 | 平均检出率 | |||||
---|---|---|---|---|---|---|
Decoder参数1 | Decoder参数2 | Decoder参数4 | Decoder参数6 | Decoder参数8 | Decoder参数10 | |
2 | 41.1 | 44.6 | 46.5 | 50.2 | 53.1 | 52.2 |
4 | 42.3 | 47.5 | 49.2 | 51.9 | 54.7 | 52.6 |
6 | 44.5 | 47.9 | 51.8 | 54.3 | 55.6 | 53.2 |
8 | 44.9 | 48.7 | 53.1 | 52.6 | 54.8 | 51.0 |
10 | 44.6 | 50.4 | 53.5 | 54.0 | 52.4 | 49.9 |
Table 3 Average detection rate of models corresponding to different Encoder and Decoder parameters (%)
Encoder参数 | 平均检出率 | |||||
---|---|---|---|---|---|---|
Decoder参数1 | Decoder参数2 | Decoder参数4 | Decoder参数6 | Decoder参数8 | Decoder参数10 | |
2 | 41.1 | 44.6 | 46.5 | 50.2 | 53.1 | 52.2 |
4 | 42.3 | 47.5 | 49.2 | 51.9 | 54.7 | 52.6 |
6 | 44.5 | 47.9 | 51.8 | 54.3 | 55.6 | 53.2 |
8 | 44.9 | 48.7 | 53.1 | 52.6 | 54.8 | 51.0 |
10 | 44.6 | 50.4 | 53.5 | 54.0 | 52.4 | 49.9 |
典型金具 | 正常 | 缺垫片 | 缺螺母 | 缺销钉 |
---|---|---|---|---|
挂点 | 1349 | 312 | 41 | 109 |
防震锤 | 210 | 217 | 12 | 37 |
绝缘子 | 356 | 112 | 26 | 75 |
间隔棒 | 145 | 15 | 6 | 4 |
Table 4 Statistical table of the number of samples of various types of bolts on different fittings
典型金具 | 正常 | 缺垫片 | 缺螺母 | 缺销钉 |
---|---|---|---|---|
挂点 | 1349 | 312 | 41 | 109 |
防震锤 | 210 | 217 | 12 | 37 |
绝缘子 | 356 | 112 | 26 | 75 |
间隔棒 | 145 | 15 | 6 | 4 |
金具部件位置 | 正常状态下螺栓元件组成 |
---|---|
挂点 | 螺母+销钉或垫片+销钉 |
防震锤 | 螺母+销钉或垫片+销钉 |
绝缘子 | 螺母+销钉 |
间隔棒 | 垫片+螺母+销钉或垫片+销钉 |
Table 5 The composition of the components in the normal state of the bolts on different fittings
金具部件位置 | 正常状态下螺栓元件组成 |
---|---|
挂点 | 螺母+销钉或垫片+销钉 |
防震锤 | 螺母+销钉或垫片+销钉 |
绝缘子 | 螺母+销钉 |
间隔棒 | 垫片+螺母+销钉或垫片+销钉 |
图像 | 先验知识 | 标签信息 |
---|---|---|
挂点 主视图 | ls-zc | |
防震锤 俯视图 | ls-qdp | |
挂点 俯视图 | ls-qlm | |
间隔棒 主视图 | ls-qxd |
Table 6 Partial bolt defect image and corresponding label example
图像 | 先验知识 | 标签信息 |
---|---|---|
挂点 主视图 | ls-zc | |
防震锤 俯视图 | ls-qdp | |
挂点 俯视图 | ls-qlm | |
间隔棒 主视图 | ls-qxd |
方法 | mAP (%) | FPS (s) | Weights (MB) |
---|---|---|---|
YOLOv5l | 57.9 | 25.7 | 87.33 |
Faster R-CNN | 65.4 | 6.9 | 141.47 |
DETR | 67.8 | 7.6 | 126.07 |
改进DETR | 70.6 | 5.5 | 174.72 |
Table 7 Comparison results of different models
方法 | mAP (%) | FPS (s) | Weights (MB) |
---|---|---|---|
YOLOv5l | 57.9 | 25.7 | 87.33 |
Faster R-CNN | 65.4 | 6.9 | 141.47 |
DETR | 67.8 | 7.6 | 126.07 |
改进DETR | 70.6 | 5.5 | 174.72 |
组别 | 方法 | mAP (%) | FPS (s) |
---|---|---|---|
1 | DETR | 67.8 | 7.6 |
2 | + CILLF | 69.3 | 7.1 |
3 | + CILLF +V-K Attention | 70.6 | 5.5 |
Table 8 Comprehensive use of different improvement strategies to compare the detection effects
组别 | 方法 | mAP (%) | FPS (s) |
---|---|---|---|
1 | DETR | 67.8 | 7.6 |
2 | + CILLF | 69.3 | 7.1 |
3 | + CILLF +V-K Attention | 70.6 | 5.5 |
Fig. 7 Comparison of detection results between mainstream target detection algorithms and improved DETR model ((a) Faster R-CNN; (b) YOLOv5l; (c) Improved DETR)
[1] | 赵振兵, 张薇, 翟永杰, 等. 电力视觉技术的概念、研究现状与展望[J]. 电力科学与工程, 2020, 36(1): 1-8. |
ZHAO Z B, ZHANG W, ZHAI Y J, et al. Concept, research status and prospect of electric power vision technology[J]. Electric Power Science and Engineering, 2020, 36(1): 1-8. (in Chinese) | |
[2] | ZHOU X Y, ZHUO J C, KRÄHENBÜHL P. Bottom-up object detection by grouping extreme and center points[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 850-859. |
[3] | 孙宗康, 饶睦敏, 曹裕灵, 等. 基于小样本不均衡数据的供水管道泄漏智能检测算法[J]. 图学学报, 2022, 43(5): 825-831. |
SUN Z K, RAO M M, CAO Y L, et al. Water supply pipeline leakage intelligent detection algorithm based on small and unbalanced data[J]. Journal of Graphics, 2022, 43(5): 825-831. (in Chinese) | |
[4] | 奚川龙. 基于路侧毫米波雷达和相机信息融合的车辆目标检测研究[D]. 重庆: 重庆邮电大学, 2021. |
XI C L. Research on vehicle target detection based on information fusion of roadside millimeter wave radar and camera[D]. Chongqing: Chongqing University of Posts and Telecommunications, 2021. (in Chinese) | |
[5] | 赵振兵, 蒋志钢, 李延旭, 等. 输电线路部件视觉缺陷检测综述[J]. 中国图象图形学报, 2021, 26(11): 2545-2560. |
ZHAO Z B, JIANG Z G, LI Y X, et al. Overview of visual defect detection of transmission line components[J]. Journal of Image and Graphics, 2021, 26(11): 2545-2560. (in Chinese) | |
[6] | REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 779-788. |
[7] | 谷雨, 赵军. 列车闸瓦钎及闸瓦故障图像检测算法研究[J/OL]. (2022-07-20) [2022-08-08]. http://kns.cnki.net/kcms/det-ail/10.1034.T.20220719.1607.002.html. |
GU Y, ZHAO J. Research on image detection algorithm of freight train brake shoe bolt and brake shoe fault[J/OL]. (2022-07-20) [2022-08-08]. http://kns.cnki.net/kcms/det-ail/10.1034.T.20220719.1607.002.html. (in Chinese) | |
[8] | LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[M]//Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 21-37. |
[9] |
YI J R, WU P X, METAXAS D N. ASSD: attentive single shot multibox detector[J]. Computer Vision and Image Understanding, 2019, 189: 102827.
DOI URL |
[10] | 赵振兵, 李延旭, 甄珍, 等. 结合KL散度和形状约束的Faster R-CNN典型金具检测方法[J]. 高电压技术, 2020, 46(9): 3018-3026. |
ZHAO Z B, LI Y X, ZHEN Z, et al. Typical fittings detection method with faster R-CNN combining KL divergence and shape constraints[J]. High Voltage Engineering, 2020, 46(9): 3018-3026. (in Chinese) | |
[11] | 翟永杰, 杨旭, 赵振兵, 等. 融合共现推理的Faster R-CNN输电线路金具检测[J]. 智能系统学报, 2021, 16(2): 237-246. |
ZHAI Y J, YANG X, ZHAO Z B, et al. Integrating co-occurrence reasoning for Faster R-CNN transmission line fitting detection[J]. CAAI Transactions on Intelligent Systems, 2021, 16(2): 237-246. (in Chinese) | |
[12] | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]// The 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010. |
[13] | ZHU X Z, SU W J, LU L W, et al. Deformable DETR: deformable transformers for end-to-end object detection[EB/OL]. [2022-07-15]. https://arxiv.org/abs/2010.04159. |
[14] | 张乃雪, 钟羽中, 赵涛, 等. 基于Smooth-DETR的产品表面小尺寸缺陷检测算法[J]. 计算机应用研究, 2022, 39(8): 2520-2525. |
ZHANG N X, ZHONG Y Z, ZHAO T, et al. Detection method for small-size defects based on Smooth-DETR[J]. Application Research of Computers, 2022, 39(8): 2520-2525. (in Chinese) | |
[15] | 熊怡梦. 基于标签语义和Transformer的元学习小样本目标检测方法研究[D]. 西安: 西安电子科技大学, 2021. |
XIONG Y M. Label semantics and Transformer for meta learning few-shot object detection[D]. Xi′an: Xidian University, 2021. (in Chinese) | |
[16] | LI F, ZHANG H, LIU S L, et al. DN-DETR: accelerate DETR training by introducing query DeNoising[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 13609-13617. |
[17] | CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[M]//Computer Vision - ECCV 2020. Cham: Springer International Publishing, 2020: 213-229. |
[18] | LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 9992-10002. |
[19] |
孟晓娟, 张月琴, 郝晓丽, 等. 多分类深度卷积生成对抗网络的皮带撕裂检测[J]. 计算机工程与应用, 2021, 57(16): 269-275.
DOI |
MENG X J, ZHANG Y Q, HAO X L, et al. Multi-class deep convolutional generative adversarial networks for belt tear detection[J]. Computer Engineering and Applications, 2021, 57(16): 269-275. (in Chinese)
DOI |
|
[20] | 李军, 李明, 曾蒸, 等. 基于改进激活函数和学习率的双向LSTM研究[J]. 重庆师范大学学报: 自然科学版, 2021, 38(2): 70-76. |
LI J, LI M, ZENG Z, et al. Research on bidirectional LSTM based on improved activation function and learning rate[J]. Journal of Chongqing Normal University: Natural Science, 2021, 38(2): 70-76. (in Chinese) | |
[21] | 何颖宣. 基于多标签学习的螺栓多属性分类方法研究[D]. 北京: 华北电力大学, 2021. |
HE Y X. Research on bolt multi-attribute classification method based on multi-label learning[D]. Beijing: North China Electric Power University, 2021. (in Chinese) |
[1] |
HAO Shuai, ZHAO Xin-sheng, MA Xu, ZHANG Xu, HE Tian, HOU Li-xiang.
Multi-class defect target detection method for transmission lines
based on TR-YOLOv5
[J]. Journal of Graphics, 2023, 44(4): 667-676.
|
[2] |
PAN Dong-hui, JIN Ying-han, SUN Xu, LIU Yu-sheng, ZHANG Dong-liang.
CTH-Net: CNN-Transformer hybrid network for garment image
generation from sketches and color points
[J]. Journal of Graphics, 2023, 44(1): 120-130.
|
[3] | WANG Yu-ping , ZENG Yi , LI Sheng-hui , ZHANG Lei. A Transformer-based 3D human pose estimation method [J]. Journal of Graphics, 2023, 44(1): 139-145. |
[4] | HU Hai-tao , DU Hao-chen , WANG Su-qin , SHI Min , ZHU Deng-ming, . Improved YOLOX method for detecting surface defects of drug blister aluminum foil [J]. Journal of Graphics, 2022, 43(5): 803-814. |
[5] | CHANG Shengjie1,2, DANG Jianwu1,2, WANG Yangping1,2. BIM Model and Information Integration of Traction Substation Equipment Based on Revit [J]. Journal of Graphics, 2018, 39(4): 771-777. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||