基于深度学习的电力设备铭牌文本检测方法

doi:10.11996/JG.j.2095-302X.2023040691

图学学报 ›› 2023, Vol. 44 ›› Issue (4): 691-698.DOI: 10.11996/JG.j.2095-302X.2023040691

• 图像处理与计算机视觉 • 上一篇下一篇

基于深度学习的电力设备铭牌文本检测方法

王道累(), 康博, 朱瑞()

上海电力大学能源与机械工程学院，上海 200090

收稿日期:2022-11-08 接受日期:2023-01-12 出版日期:2023-08-31 发布日期:2023-08-16
通讯作者: 朱瑞(1981-)，女，副教授，博士。主要研究方向为绝缘子检测、输变电设备故障检测和深度学习。E-mail：zhuruish@163.com
作者简介:
王道累(1981-)，男，教授，博士。主要研究方向为计算机视觉、图像处理和CAD/CAM。E-mail：alfredwdl@shiep.edu.cn
基金资助:
国家自然科学基金项目(61502297)

Text detection method for electrical equipment nameplates based on deep learning

WANG Dao-lei(), KANG Bo, ZHU Rui()

College of Energy and Mechanical Engineering, Shanghai University of Electric Power, Shanghai 200090, China

Received:2022-11-08 Accepted:2023-01-12 Online:2023-08-31 Published:2023-08-16
Contact: ZHU Rui (1981-), associate professor. Ph.D. Her main research interests cover insulator detection, transformation equipment fault detection and deep learning. E-mail：zhuruish@163.com
About author:
WANG Dao-lei (1981-), professor, Ph.D. His main research interests cover computer vision, image processing and CAD/CAM. E-mail：alfredwdl@shiep.edu.cn
Supported by:
National Natural Science Foundation of China(61502297)

摘要/Abstract

摘要：

电力设备铭牌的快速检测可以帮助变电站、电厂了解设备信息，进行定期检修与维护，以保证设备的正常运行。针对目前的文本检测网络无法做到提高精确率的同时兼顾检测效率的问题，提出了在DBNet网络模型中引入注意力模块(CBAM)，并改进检测头，在主干网络中引入多尺度特征金字塔(FPN)结构，并在原始的FPN上进行改进。针对目前电力设备铭牌并无公开数据集且较难采集数据的情况，提出了将数张铭牌图片裁剪成矩形，再以一定比例进行拼接成新的图像的数据增强方法，以此对数据集进行了有效地扩充。实验结果表明，数据增强方法和改进后的DBNet网络结构在检测性能上均有提升，优于目前大多数文本检测网络结构。改进后的DBNet网络结构检测精确率达到了90.3%，召回率达到了79.7%，F值达到了84.7%，相较于原始模型，F值提升了3.3个百分点。在检测速度变化损失很小的同时，极大地提高了检测性能。

关键词: 文本检测, DBNet, 注意力模块, 数据增强, 电力设备铭牌

Abstract:

The prompt detection of power equipment nameplates can help the complete transformer substations and power plants to efficiently comprehend device information and perform necessary maintenance, thus ensuring the proper functioning. This thesis addressed the problem of enhancing text detection efficiency while also taking into account the improvement of precision. To that end, we introduced the concept of convolutional block attention module (CBAM) into the DBNet, and improved the detection head. Multi-scale feature feature pyramid networks (FPN) structures were introduced into the backbone network, improving upon the original FPN. Meanwhile, in view of the absence of public data for power equipment nameplates and difficulties in obtaining it, we proposed a technique to enhance the data by cutting nameplate images into rectangles and then splicing them together into a new image. In this way, the data set could be effectively expanded. The experimental results showed that both the data enhancement method and the improved DBNet network structure proposed in this paper have played a role in improving the detection performance, surpassing most current text detection network structures on the market. The improved DBNet network structure combined with data enhancement method yielded a precision rate of 90.3% and a recalling rate of 79.7%. The rate of F-measure also increased to nearly 84.7%, a 3.3% improvement over the original model, indicating that the detection performance was greatly improved while the loss of detecting speed changes remained minimal.

Key words: text detection, DBNet, CBAM, data enhancement, electrical equipment nameplates

中图分类号:

TP391.1

王道累, 康博, 朱瑞. 基于深度学习的电力设备铭牌文本检测方法[J]. 图学学报, 2023, 44(4): 691-698.

WANG Dao-lei, KANG Bo, ZHU Rui. Text detection method for electrical equipment nameplates based on deep learning[J]. Journal of Graphics, 2023, 44(4): 691-698.

图/表 13

图1 网络结构

Fig. 1 Model structure

图2 数据增强示例((a)原始图片；(b)增强图片)

Fig. 2 Example of data augmentation ((a) Original images; (b) Enhanced images)

图3 CBAM模块整体结构

Fig. 3 The structure of CBAM

图4 检测头整体结构

Fig. 4 The structure of detection head

图5 改进后网络结构

Fig. 5 The structure of improved model

图6 铭牌类型((a)强反光；(b)镜面；(c)覆盖物)

Fig. 6 Nameplate type ((a) Reflect light; (b) Monute surface; (c) Overlap)

表1 不同模型的公共数据集检测结果

Table 1 Public dataset detection results for different models

网络模型	Datasets	P(%)	R(%)	F(%)	FPS
DBNet	ICDAR2013	96.3	90.6	93.4	27.1
DBNet	ICDAR2015	93.7	86.1	89.7	25.8
PSENet	ICDAR2013	94.6	89.4	91.9	17.3
PSENet	ICDAR2015	89.3	85.2	87.2	15.7
EAST	ICDAR2013	89.4	83.1	86.1	57.6
EAST	ICDAR2015	83.3	78.3	81.0	53.1
Ours	ICDAR2013	97.5	91.1	94.2	23.8
Ours	ICDAR2015	94.1	87.3	90.6	21.5

表2 不同模型的电力铭牌检测结果

Table 2 The results of different power nameplate detection algorithms

网络模型	P(%)	R(%)	F(%)	FPS
DBNet	87.3	76.4	81.4	16.7
PSEnet	83.7	74.3	78.7	9.1
EAST	77.6	69.4	73.3	31.7
ABCNet	88.6	79.3	83.7	14.0
SwinTextSpotter	83.4	72.5	77.6	19.7
Ours	90.3	79.7	84.7	13.1

图7 原始模型与改进模型检测结果对比((a) DBNet检测结果示例；(b)改进后网络检测结果示例)

Fig. 7 Comparison of detection results between original model and improved model ((a) DBNet test result; (b) Ours test result)

表3 数据增强消融实验结果(%)

Table 3 The results of data enhanced ablation experiments (%)

数据增强方式	P	R	F
无数据增强	87.3	76.4	81.4
Ours	88.1	77.2	82.2

表4 注意力机制消融实验结果

Table 4 The results of CBAM experiments

模型	P(%)	R(%)	F(%)	FPS
DBNet	87.3	76.4	81.4	16.7
DBNet+CBAM	88.6	77.3	82.6	14.4

表5 特征金字塔消融实验结果

Table 5 The results of FPN experiments

模型	P(%)	R(%)	F(%)	FPS
DBNet	87.3	76.4	81.4	16.7
Ours	88.6	77.9	82.9	14.3

表6 主干网络消融实验结果

Table 6 The results of backbone network ablation experiments

主干网络	P(%)	R(%)	F(%)	FPS
ResNet18	83.4	75.3	79.1	24.9
ResNet50	90.3	79.7	84.7	13.1
MobilnetV3	86.5	76.1	81.0	27.0
ShuffleNetV2	84.7	75.3	79.7	29.3

参考文献 18

[1]	ZHOU X Y, YAO C, WEN H, et al. EAST: an efficient and accurate scene text detector[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 2642-2651.
[2]	陈晓龙, 陈显龙, 袁建平, 等. 基于深度学习的电力设备铭牌识别[J]. 广西大学学报: 自然科学版, 2018, 43(6): 2216-2226.
	CHEN X L, CHEN X L, YUAN J P, et al. Electricity equipment nameplate recognition based on deep learning[J]. Journal of Guangxi University: Natural Science Edition, 2018, 43(6): 2216-2226 (in Chinese).
[3]	GUO C J, MA X L, SHU Y, et al. Text detection of power equipment nameplates based on deep learning[C]// 2019 IEEE 9th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems. New York: IEEE Press, 2020: 1635-1639.
[4]	LIAO M H, SHI B G, BAI X, et al. TextBoxes: a fast text detector with a single deep neural network[EB/OL]. [2022-05-10]. https://arxiv.org/abs/1611.06779v1.
[5]	ZHU Y X, DU J. Sliding line point regression for shape robust scene text detection[C]// The 24th International Conference on Pattern Recognition. New York: IEEE Press, 2018: 3735-3740.
[6]	SHI B G, BAI X, BELONGIE S. Detecting oriented text in natural images by linking segments[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 3482-3490.
[7]	MA J Q, SHAO W Y, YE H, et al. Arbitrary-oriented scene text detection via rotation proposals[J]. IEEE Transactions on Multimedia, 2018, 20(11): 3111-3122. DOI URL
[8]	BAEK Y, LEE B, HAN D, et al. Character region awareness for text detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9357-9366.
[9]	XIE E Z, ZANG Y H, SHAO S, et al. Scene text detection with supervised pyramid context network[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 9038-9045. DOI URL
[10]	LONG S B, RUAN J Q, ZHANG W J, et al. TextSnake: a flexible representation for detecting text of arbitrary shapes[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 19-35.
[11]	WANG W H, XIE E Z, LI X, et al. Shape robust text detection with progressive scale expansion network[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9328-9337.
[12]	LIU Y L, CHEN H, SHEN C H, et al. ABCNet: real-time scene text spotting with adaptive bezier-curve network[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9806-9815.
[13]	HUANG M X, LIU Y L, PENG Z H, et al. SwinTextSpotter: scene text spotting via better synergy between text detection and text recognition[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 4583-4593.
[14]	LIAO M H, ZOU Z S, WAN Z Y, et al. Real-time scene text detection with differentiable binarization and adaptive scale fusion[C]// IEEE Transactions on Pattern Analysis and Machine Intelligence. New York: IEEE Press, 2022: 919-931.
[15]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]//Computer Vision - ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[16]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[17]	HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 1314-1324.
[18]	MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]// Computer Vision - ECCV 2018:15th European Conference. New York: ACM, 2018: 122-138.

基于深度学习的电力设备铭牌文本检测方法

Text detection method for electrical equipment nameplates based on deep learning

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 18

相关文章 13

编辑推荐

Metrics

本文评价

[1]	阎光伟, 刘润泽, 焦润海, 何慧. 基于改进Cascade RCNN的输电线路防振锤脱落检测方法 [J]. 图学学报, 2023, 44(5): 849-860.
[2]	曹义亲, 周一纬, 徐露. 基于E-YOLOX的实时金属表面缺陷检测算法[J]. 图学学报, 2023, 44(4): 677-690.
[3]	李雨, 闫甜甜, 周东生, 魏小鹏. 基于注意力机制与深度多尺度特征融合的自然场景文本检测[J]. 图学学报, 2023, 44(3): 473-481.
[4]	曾武, 朱恒亮, 邢树礼, 林江宏, 毛国君. 显著性检测引导的图像数据增强方法[J]. 图学学报, 2023, 44(2): 260-270.
[5]	皮骏, 刘宇恒, 李久昊. 基于YOLOv5s的轻量化森林火灾检测算法研究[J]. 图学学报, 2023, 44(1): 26-32.
[6]	范震, 刘晓静, 李小波, 崔亚超. 一种对光照和遮挡鲁棒的单应性估计方法[J]. 图学学报, 2023, 44(1): 166-176.
[7]	赵辉 , 赵尧 , 金林林 , 董兰芳 , 肖潇 . 基于 YOLOX 的小目标烟火检测技术研究与实现[J]. 图学学报, 2022, 43(5): 783-790.
[8]	孙宗康 , 饶睦敏 , 曹裕灵 , 史艳丽 . 基于小样本不均衡数据的供水管道泄漏智能检测算法 [J]. 图学学报, 2022, 43(5): 825-831.
[9]	安峰 , 戴军 , 韩振 , 严仲兴 . 引入注意力机制的自监督光流计算[J]. 图学学报, 2022, 43(5): 841-848.
[10]	陈昭俊, 储珺, 曾伦杰. 基于动态加权类别平衡损失的多类别口罩佩戴检测[J]. 图学学报, 2022, 43(4): 590-598.
[11]	方洪波, 万广, 陈忠辉, 黄以卫, 张文勇, 谢本亮. 基于改进 YOLOv5s 的离线手写数学符号识别[J]. 图学学报, 2022, 43(3): 387-395.
[12]	蒋圣南，陈恩庆，郑铭耀，段建康 . 基于 ResNeXt 的人体动作识别[J]. 图学学报, 2020, 41(2): 277-282.
[13]	蒋杰，熊昌镇. 一种数据增强和多模型集成的细粒度分类算法[J]. 图学学报, 2018, 39(2): 244-250.